[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056586788339-29547-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:49:46 dist-test-slave-1xrh krb5kdc[29555](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:49:46 dist-test-slave-1xrh krb5kdc[29555](info): set up 2 sockets
Aug 06 22:49:46 dist-test-slave-1xrh krb5kdc[29555](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260806 22:49:47.876789 29547 mini_kdc.cc:121] Time spent starting KDC: real 1.070s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056586788339-29547-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056586788339-29547-0/krb5kdc/test-user.keytab.
Aug 06 22:49:47 dist-test-slave-1xrh krb5kdc[29555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056587, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:49:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:49:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.218.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.218.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.218.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab.
Entry for principal kudu/127.28.218.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab.
I20260806 22:49:47.994663 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056586788339-29547-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056586788339-29547-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056586788339-29547-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.218.254:40675
--webserver_interface=127.28.218.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab
--principal=kudu/127.28.218.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.218.212:40377
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.218.254:40675
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:49:48.077270 29568 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:49:48.077466 29568 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:49:48.077487 29568 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:49:48.078572 29568 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:49:48.078608 29568 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:49:48.078622 29568 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:49:48.078634 29568 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:49:48.078647 29568 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:49:48.080180 29568 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:40377
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056586788339-29547-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056586788339-29547-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.218.254:40675
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab
--principal=kudu/127.28.218.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.218.254:40675
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.218.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29568
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056586788339-29547-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:49:48.080363 29568 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:49:48.080523 29568 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:49:48.085659 29573 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:49:48.085724 29568 server_base.cc:1034] running on GCE node
W20260806 22:49:48.085682 29576 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:48.085682 29574 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:49:48.086176 29568 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:49:48.086572 29568 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:49:48.087718 29568 hybrid_clock.cc:648] HybridClock initialized: now 1786056588087709 us; error 60 us; skew 500 ppm
Aug 06 22:49:48 dist-test-slave-1xrh krb5kdc[29555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056588, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:49:48.090204 29568 init.cc:377] Logged in from keytab as kudu/127.28.218.254@KRBTEST.COM (short username kudu)
I20260806 22:49:48.090662 29568 webserver.cc:460] Webserver started at http://127.28.218.254:40709/ using document root <none> and password file <none>
I20260806 22:49:48.090878 29568 fs_manager.cc:359] Metadata directory not provided
I20260806 22:49:48.090929 29568 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:49:48.091109 29568 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:49:48.092231 29568 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056586788339-29547-0/minicluster-data/master-0/data/instance:
uuid: "e3cd145602754fbdb8a6e5e5a0598e3e"
format_stamp: "Formatted at 2026-08-06 22:49:48 on dist-test-slave-1xrh"
server_key: "be619ee533cd9281252e59a6e3f00080"
server_key_iv: "c8adb1d9ebbbf80c37d79073c1f8fc5f"
server_key_version: "encryptionkey@0"
I20260806 22:49:48.092602 29568 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056586788339-29547-0/minicluster-data/master-0/wal/instance:
uuid: "e3cd145602754fbdb8a6e5e5a0598e3e"
format_stamp: "Formatted at 2026-08-06 22:49:48 on dist-test-slave-1xrh"
server_key: "be619ee533cd9281252e59a6e3f00080"
server_key_iv: "c8adb1d9ebbbf80c37d79073c1f8fc5f"
server_key_version: "encryptionkey@0"
I20260806 22:49:48.094069 29568 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:49:48.094854 29583 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:48.095113 29568 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:49:48.095276 29568 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056586788339-29547-0/minicluster-data/master-0/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056586788339-29547-0/minicluster-data/master-0/wal
uuid: "e3cd145602754fbdb8a6e5e5a0598e3e"
format_stamp: "Formatted at 2026-08-06 22:49:48 on dist-test-slave-1xrh"
server_key: "be619ee533cd9281252e59a6e3f00080"
server_key_iv: "c8adb1d9ebbbf80c37d79073c1f8fc5f"
server_key_version: "encryptionkey@0"
I20260806 22:49:48.095355 29568 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056586788339-29547-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056586788339-29547-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056586788339-29547-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:49:48.101758 29568 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:49:48.104063 29568 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:49:48.104230 29568 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:49:48.108417 29568 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.254:40675
I20260806 22:49:48.108470 29637 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.254:40675 every 8 connection(s)
I20260806 22:49:48.108839 29568 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
I20260806 22:49:48.109805 29638 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:49:48.112087 29638 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3cd145602754fbdb8a6e5e5a0598e3e: Bootstrap starting.
I20260806 22:49:48.112669 29638 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e3cd145602754fbdb8a6e5e5a0598e3e: Neither blocks nor log segments found. Creating new log.
I20260806 22:49:48.112963 29638 log.cc:824] T 00000000000000000000000000000000 P e3cd145602754fbdb8a6e5e5a0598e3e: Log is configured to *not* fsync() on all Append() calls
I20260806 22:49:48.113709 29638 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3cd145602754fbdb8a6e5e5a0598e3e: No bootstrap required, opened a new log
I20260806 22:49:48.115291 29638 raft_consensus.cc:348] T 00000000000000000000000000000000 P e3cd145602754fbdb8a6e5e5a0598e3e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3cd145602754fbdb8a6e5e5a0598e3e" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 40675 } }
I20260806 22:49:48.115442 29638 raft_consensus.cc:374] T 00000000000000000000000000000000 P e3cd145602754fbdb8a6e5e5a0598e3e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:49:48.115521 29638 raft_consensus.cc:729] T 00000000000000000000000000000000 P e3cd145602754fbdb8a6e5e5a0598e3e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3cd145602754fbdb8a6e5e5a0598e3e, State: Initialized, Role: FOLLOWER
I20260806 22:49:48.115715 29638 consensus_queue.cc:260] T 00000000000000000000000000000000 P e3cd145602754fbdb8a6e5e5a0598e3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3cd145602754fbdb8a6e5e5a0598e3e" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 40675 } }
I20260806 22:49:48.115854 29638 raft_consensus.cc:388] T 00000000000000000000000000000000 P e3cd145602754fbdb8a6e5e5a0598e3e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:49:48.115900 29638 raft_consensus.cc:482] T 00000000000000000000000000000000 P e3cd145602754fbdb8a6e5e5a0598e3e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:49:48.115949 29638 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e3cd145602754fbdb8a6e5e5a0598e3e [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:49:48.116477 29638 raft_consensus.cc:504] T 00000000000000000000000000000000 P e3cd145602754fbdb8a6e5e5a0598e3e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3cd145602754fbdb8a6e5e5a0598e3e" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 40675 } }
I20260806 22:49:48.116595 29638 leader_election.cc:302] T 00000000000000000000000000000000 P e3cd145602754fbdb8a6e5e5a0598e3e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e3cd145602754fbdb8a6e5e5a0598e3e; no voters:
I20260806 22:49:48.116775 29638 leader_election.cc:288] T 00000000000000000000000000000000 P e3cd145602754fbdb8a6e5e5a0598e3e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:49:48.116811 29641 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e3cd145602754fbdb8a6e5e5a0598e3e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:49:48.116990 29641 raft_consensus.cc:686] T 00000000000000000000000000000000 P e3cd145602754fbdb8a6e5e5a0598e3e [term 1 LEADER]: Becoming Leader. State: Replica: e3cd145602754fbdb8a6e5e5a0598e3e, State: Running, Role: LEADER
I20260806 22:49:48.117026 29638 sys_catalog.cc:564] T 00000000000000000000000000000000 P e3cd145602754fbdb8a6e5e5a0598e3e [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:49:48.117080 29641 consensus_queue.cc:237] T 00000000000000000000000000000000 P e3cd145602754fbdb8a6e5e5a0598e3e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3cd145602754fbdb8a6e5e5a0598e3e" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 40675 } }
I20260806 22:49:48.117501 29642 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3cd145602754fbdb8a6e5e5a0598e3e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e3cd145602754fbdb8a6e5e5a0598e3e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3cd145602754fbdb8a6e5e5a0598e3e" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 40675 } } }
I20260806 22:49:48.117586 29642 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3cd145602754fbdb8a6e5e5a0598e3e [sys.catalog]: This master's current role is: LEADER
I20260806 22:49:48.117745 29643 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3cd145602754fbdb8a6e5e5a0598e3e [sys.catalog]: SysCatalogTable state changed. Reason: New leader e3cd145602754fbdb8a6e5e5a0598e3e. Latest consensus state: current_term: 1 leader_uuid: "e3cd145602754fbdb8a6e5e5a0598e3e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3cd145602754fbdb8a6e5e5a0598e3e" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 40675 } } }
I20260806 22:49:48.117925 29643 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3cd145602754fbdb8a6e5e5a0598e3e [sys.catalog]: This master's current role is: LEADER
I20260806 22:49:48.117992 29648 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:49:48.118535 29648 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:49:48.119347 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 29568
I20260806 22:49:48.119463 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056586788339-29547-0/minicluster-data/master-0/wal/instance
I20260806 22:49:48.120203 29648 catalog_manager.cc:1347] Generated new cluster ID: 24caa5849f374d81a8fa3eec2b2ad22a
I20260806 22:49:48.120246 29648 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:49:48.120324 29547 external_mini_cluster.cc:1428] Setting key 944bb4cf19e7b8ab0f04738cc9da2aaa
I20260806 22:49:48.125897 29648 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:49:48.126816 29648 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:49:48.133246 29648 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e3cd145602754fbdb8a6e5e5a0598e3e: Generated new TSK 0
I20260806 22:49:48.133396 29648 catalog_manager.cc:1514] Initializing in-progress tserver states...
Aug 06 22:49:48 dist-test-slave-1xrh krb5kdc[29555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056587, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.218.254@KRBTEST.COM
I20260806 22:49:48.141696 29662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:48.121150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51804 (local address 127.28.218.254:40675)
0806 22:49:48.125570 (+ 4420us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:48.125579 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:48.126112 (+ 533us) server_negotiation.cc:407] Connection header received
0806 22:49:48.129105 (+ 2993us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:48.129110 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:48.129150 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:48.129283 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:48.131120 (+ 1837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:48.132041 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:48.133485 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:48.133918 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:48.134035 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:48.137693 (+ 3658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:48.137699 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:48.137699 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:49:48.137713 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:48.139573 (+ 1860us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:48.139988 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:48.139991 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:48.139991 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:48.140081 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:48.140346 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:48.140347 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:48.140347 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:48.140646 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:48.140846 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:48.141122 (+ 276us) server_negotiation.cc:299] Negotiation successful
0806 22:49:48.141294 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4370,"thread_start_us":78,"threads_started":1}
WARNING: no policy specified for kudu/127.28.218.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.218.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.218.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.193.keytab.
Entry for principal kudu/127.28.218.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.193.keytab.
I20260806 22:49:48.170547 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056586788339-29547-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056586788339-29547-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056586788339-29547-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056586788339-29547-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.218.193:0
--local_ip_for_outbound_sockets=127.28.218.193
--webserver_interface=127.28.218.193
--webserver_port=0
--tserver_master_addrs=127.28.218.254:40675
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.193.keytab
--principal=kudu/127.28.218.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.218.212:40377
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:49:48.282712 29666 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:49:48.282907 29666 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:49:48.282931 29666 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:49:48.284072 29666 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:49:48.284113 29666 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:49:48.284140 29666 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.218.193
I20260806 22:49:48.285559 29666 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:40377
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056586788339-29547-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056586788339-29547-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.193.keytab
--principal=kudu/127.28.218.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.218.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056586788339-29547-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.218.193
--webserver_port=0
--tserver_master_addrs=127.28.218.254:40675
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29666
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.218.193
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056586788339-29547-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:49:48.285777 29666 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:49:48.285954 29666 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:49:48.291918 29674 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:48.292032 29672 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:49:48.292160 29666 server_base.cc:1034] running on GCE node
W20260806 22:49:48.292207 29671 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:49:48.292413 29666 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:49:48.292668 29666 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:49:48.293843 29666 hybrid_clock.cc:648] HybridClock initialized: now 1786056588293815 us; error 39 us; skew 500 ppm
Aug 06 22:49:48 dist-test-slave-1xrh krb5kdc[29555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056588, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:49:48.297078 29666 init.cc:377] Logged in from keytab as kudu/127.28.218.193@KRBTEST.COM (short username kudu)
I20260806 22:49:48.297472 29666 webserver.cc:460] Webserver started at http://127.28.218.193:35861/ using document root <none> and password file <none>
I20260806 22:49:48.297660 29666 fs_manager.cc:359] Metadata directory not provided
I20260806 22:49:48.297701 29666 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:49:48.297788 29666 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:49:48.298847 29666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056586788339-29547-0/minicluster-data/ts-0/data/instance:
uuid: "a480cee5cf8c4927b83f0e1a37371a30"
format_stamp: "Formatted at 2026-08-06 22:49:48 on dist-test-slave-1xrh"
server_key: "a37950de2cce46527d2ae3522d32da11"
server_key_iv: "07197913a8882deb7eb6a59a0528978a"
server_key_version: "encryptionkey@0"
I20260806 22:49:48.299250 29666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056586788339-29547-0/minicluster-data/ts-0/wal/instance:
uuid: "a480cee5cf8c4927b83f0e1a37371a30"
format_stamp: "Formatted at 2026-08-06 22:49:48 on dist-test-slave-1xrh"
server_key: "a37950de2cce46527d2ae3522d32da11"
server_key_iv: "07197913a8882deb7eb6a59a0528978a"
server_key_version: "encryptionkey@0"
I20260806 22:49:48.300488 29666 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:49:48.301139 29681 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:48.301303 29666 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:49:48.301389 29666 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056586788339-29547-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056586788339-29547-0/minicluster-data/ts-0/wal
uuid: "a480cee5cf8c4927b83f0e1a37371a30"
format_stamp: "Formatted at 2026-08-06 22:49:48 on dist-test-slave-1xrh"
server_key: "a37950de2cce46527d2ae3522d32da11"
server_key_iv: "07197913a8882deb7eb6a59a0528978a"
server_key_version: "encryptionkey@0"
I20260806 22:49:48.301461 29666 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056586788339-29547-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056586788339-29547-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056586788339-29547-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:49:48.318359 29666 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:49:48.320349 29666 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:49:48.320531 29666 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:49:48.320748 29666 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:49:48.321060 29666 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:49:48.321134 29666 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:48.321188 29666 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:49:48.321216 29666 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:48.327356 29666 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.193:34039
I20260806 22:49:48.327392 29797 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.193:34039 every 8 connection(s)
I20260806 22:49:48.327780 29666 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056586788339-29547-0/minicluster-data/ts-0/data/info.pb
Aug 06 22:49:48 dist-test-slave-1xrh krb5kdc[29555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056588, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.193@KRBTEST.COM for kudu/127.28.218.254@KRBTEST.COM
I20260806 22:49:48.336179 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 29666
I20260806 22:49:48.336282 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056586788339-29547-0/minicluster-data/ts-0/wal/instance
I20260806 22:49:48.336473 29547 external_mini_cluster.cc:1428] Setting key 89537af406e46c785700c9780718f03b
I20260806 22:49:48.337460 29662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:48.328520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.193:39013 (local address 127.28.218.254:40675)
0806 22:49:48.328590 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:48.328593 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:48.328815 (+ 222us) server_negotiation.cc:407] Connection header received
0806 22:49:48.329287 (+ 472us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:48.329288 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:48.329298 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:48.329362 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:48.330516 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:48.331001 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:48.332020 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:48.332206 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:48.332256 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:48.334661 (+ 2405us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:48.334663 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:48.334663 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:49:48.334669 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:48.335955 (+ 1286us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:48.336435 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:48.336436 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:48.336436 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:48.336486 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:48.336782 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:48.336784 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:48.336784 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:48.336936 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:48.337005 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:48.337279 (+ 274us) server_negotiation.cc:299] Negotiation successful
0806 22:49:48.337368 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 22:49:48.337724 29800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:48.328504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:40675 (local address 127.28.218.193:39013)
0806 22:49:48.328697 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:48.328732 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:48.329204 (+ 472us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:48.329505 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:48.329508 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:48.330235 (+ 727us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:49:48.330433 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:48.330438 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:48.331104 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:48.331105 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:48.331943 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:48.331945 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:48.332279 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:48.332280 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:48.332383 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:48.332841 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:49:48.332849 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:49:48.334579 (+ 1730us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:49:48.336045 (+ 1466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:48.336053 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:48.336055 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:48.336359 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:48.336579 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:48.336580 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:48.336580 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:48.336705 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:48.336996 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:49:48.337002 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:49:48.337146 (+ 144us) client_negotiation.cc:769] Sending connection context
0806 22:49:48.337255 (+ 109us) client_negotiation.cc:240] Negotiation successful
0806 22:49:48.337413 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":107,"threads_started":1}
I20260806 22:49:48.338219 29798 heartbeater.cc:347] Connected to a master server at 127.28.218.254:40675
I20260806 22:49:48.338320 29798 heartbeater.cc:464] Registering TS with master...
I20260806 22:49:48.338505 29798 heartbeater.cc:511] Master 127.28.218.254:40675 requested a full tablet report, sending...
I20260806 22:49:48.339031 29600 ts_manager.cc:194] Registered new tserver with Master: a480cee5cf8c4927b83f0e1a37371a30 (127.28.218.193:34039)
I20260806 22:49:48.339701 29600 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.218.193@KRBTEST.COM'} at 127.28.218.193:39013
WARNING: no policy specified for kudu/127.28.218.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.218.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.218.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.194.keytab.
Entry for principal kudu/127.28.218.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.194.keytab.
I20260806 22:49:48.362286 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056586788339-29547-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056586788339-29547-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056586788339-29547-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056586788339-29547-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.218.194:0
--local_ip_for_outbound_sockets=127.28.218.194
--webserver_interface=127.28.218.194
--webserver_port=0
--tserver_master_addrs=127.28.218.254:40675
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.194.keytab
--principal=kudu/127.28.218.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.218.212:40377
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:49:48.443742 29803 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:49:48.443953 29803 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:49:48.443976 29803 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:49:48.445073 29803 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:49:48.445112 29803 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:49:48.445138 29803 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.218.194
I20260806 22:49:48.446609 29803 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:40377
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056586788339-29547-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056586788339-29547-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.194.keytab
--principal=kudu/127.28.218.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.218.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056586788339-29547-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.218.194
--webserver_port=0
--tserver_master_addrs=127.28.218.254:40675
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29803
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.218.194
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056586788339-29547-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:49:48.446841 29803 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:49:48.447052 29803 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:49:48.451840 29808 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:48.451887 29809 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:48.451896 29811 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:49:48.452047 29803 server_base.cc:1034] running on GCE node
I20260806 22:49:48.452237 29803 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:49:48.452467 29803 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:49:48.453629 29803 hybrid_clock.cc:648] HybridClock initialized: now 1786056588453608 us; error 38 us; skew 500 ppm
Aug 06 22:49:48 dist-test-slave-1xrh krb5kdc[29555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056588, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:49:48.456135 29803 init.cc:377] Logged in from keytab as kudu/127.28.218.194@KRBTEST.COM (short username kudu)
I20260806 22:49:48.456573 29803 webserver.cc:460] Webserver started at http://127.28.218.194:35375/ using document root <none> and password file <none>
I20260806 22:49:48.456787 29803 fs_manager.cc:359] Metadata directory not provided
I20260806 22:49:48.456857 29803 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:49:48.457005 29803 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:49:48.457954 29803 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056586788339-29547-0/minicluster-data/ts-1/data/instance:
uuid: "da3f21ef17ce4f749ed06886351ff63d"
format_stamp: "Formatted at 2026-08-06 22:49:48 on dist-test-slave-1xrh"
server_key: "96f83d10ada3d8aa95a31c5c18a225f8"
server_key_iv: "6f1c3f0bce6f7615013bd0ebbfd4ad28"
server_key_version: "encryptionkey@0"
I20260806 22:49:48.458313 29803 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056586788339-29547-0/minicluster-data/ts-1/wal/instance:
uuid: "da3f21ef17ce4f749ed06886351ff63d"
format_stamp: "Formatted at 2026-08-06 22:49:48 on dist-test-slave-1xrh"
server_key: "96f83d10ada3d8aa95a31c5c18a225f8"
server_key_iv: "6f1c3f0bce6f7615013bd0ebbfd4ad28"
server_key_version: "encryptionkey@0"
I20260806 22:49:48.459676 29803 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:49:48.460318 29818 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:48.460489 29803 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:49:48.460587 29803 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056586788339-29547-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056586788339-29547-0/minicluster-data/ts-1/wal
uuid: "da3f21ef17ce4f749ed06886351ff63d"
format_stamp: "Formatted at 2026-08-06 22:49:48 on dist-test-slave-1xrh"
server_key: "96f83d10ada3d8aa95a31c5c18a225f8"
server_key_iv: "6f1c3f0bce6f7615013bd0ebbfd4ad28"
server_key_version: "encryptionkey@0"
I20260806 22:49:48.460661 29803 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056586788339-29547-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056586788339-29547-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056586788339-29547-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:49:48.476344 29803 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:49:48.478546 29803 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:49:48.478714 29803 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:49:48.478920 29803 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:49:48.479274 29803 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:49:48.479310 29803 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:48.479331 29803 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:49:48.479373 29803 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:48.485039 29803 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.194:45805
I20260806 22:49:48.485116 29934 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.194:45805 every 8 connection(s)
I20260806 22:49:48.485431 29803 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056586788339-29547-0/minicluster-data/ts-1/data/info.pb
I20260806 22:49:48.486881 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 29803
I20260806 22:49:48.487004 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056586788339-29547-0/minicluster-data/ts-1/wal/instance
I20260806 22:49:48.487210 29547 external_mini_cluster.cc:1428] Setting key bcd2173a8789f280bf89367632880fd2
Aug 06 22:49:48 dist-test-slave-1xrh krb5kdc[29555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056588, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.194@KRBTEST.COM for kudu/127.28.218.254@KRBTEST.COM
I20260806 22:49:48.495718 29662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:48.486131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.194:40837 (local address 127.28.218.254:40675)
0806 22:49:48.486175 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:48.486177 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:48.486376 (+ 199us) server_negotiation.cc:407] Connection header received
0806 22:49:48.486931 (+ 555us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:48.486932 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:48.486943 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:48.487019 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:48.488249 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:48.488776 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:48.490106 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:48.490305 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:48.490378 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:48.492683 (+ 2305us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:48.492685 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:48.492685 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:49:48.492693 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:48.494339 (+ 1646us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:48.494715 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:48.494716 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:48.494717 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:48.494787 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:48.495024 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:48.495025 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:48.495025 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:48.495202 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:48.495278 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:48.495517 (+ 239us) server_negotiation.cc:299] Negotiation successful
0806 22:49:48.495604 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 22:49:48.495730 29937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:48.486100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:40675 (local address 127.28.218.194:40837)
0806 22:49:48.486269 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:48.486300 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:48.486821 (+ 521us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:48.487106 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:48.487110 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:48.487884 (+ 774us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:49:48.488159 (+ 275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:48.488165 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:48.488874 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:48.488875 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:48.490012 (+ 1137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:48.490014 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:48.490394 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:48.490395 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:48.490537 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:48.491000 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:49:48.491009 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:49:48.492612 (+ 1603us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:49:48.494429 (+ 1817us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:48.494434 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:48.494436 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:48.494641 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:48.494859 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:48.494859 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:48.494860 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:48.494947 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:48.495271 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:49:48.495275 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:49:48.495382 (+ 107us) client_negotiation.cc:769] Sending connection context
0806 22:49:48.495465 (+ 83us) client_negotiation.cc:240] Negotiation successful
0806 22:49:48.495578 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":80,"threads_started":1}
I20260806 22:49:48.496392 29935 heartbeater.cc:347] Connected to a master server at 127.28.218.254:40675
I20260806 22:49:48.496486 29935 heartbeater.cc:464] Registering TS with master...
I20260806 22:49:48.496676 29935 heartbeater.cc:511] Master 127.28.218.254:40675 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.28.218.195@KRBTEST.COM; defaulting to no policy
I20260806 22:49:48.497066 29600 ts_manager.cc:194] Registered new tserver with Master: da3f21ef17ce4f749ed06886351ff63d (127.28.218.194:45805)
I20260806 22:49:48.497432 29600 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.218.194@KRBTEST.COM'} at 127.28.218.194:40837
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.218.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.218.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.195.keytab.
Entry for principal kudu/127.28.218.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.195.keytab.
I20260806 22:49:48.514715 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056586788339-29547-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056586788339-29547-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056586788339-29547-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056586788339-29547-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.218.195:0
--local_ip_for_outbound_sockets=127.28.218.195
--webserver_interface=127.28.218.195
--webserver_port=0
--tserver_master_addrs=127.28.218.254:40675
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.195.keytab
--principal=kudu/127.28.218.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.218.212:40377
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:49:48.593828 29940 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:49:48.594056 29940 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:49:48.594095 29940 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:49:48.595269 29940 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:49:48.595357 29940 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:49:48.595414 29940 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.218.195
I20260806 22:49:48.596865 29940 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:40377
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056586788339-29547-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056586788339-29547-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.195.keytab
--principal=kudu/127.28.218.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.218.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056586788339-29547-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.218.195
--webserver_port=0
--tserver_master_addrs=127.28.218.254:40675
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29940
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.218.195
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056586788339-29547-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:49:48.597127 29940 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:49:48.597350 29940 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:49:48.602465 29946 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:48.602535 29948 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:48.602464 29945 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:49:48.602700 29940 server_base.cc:1034] running on GCE node
I20260806 22:49:48.602918 29940 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:49:48.603214 29940 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:49:48.604368 29940 hybrid_clock.cc:648] HybridClock initialized: now 1786056588604349 us; error 32 us; skew 500 ppm
Aug 06 22:49:48 dist-test-slave-1xrh krb5kdc[29555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056588, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:49:48.606760 29940 init.cc:377] Logged in from keytab as kudu/127.28.218.195@KRBTEST.COM (short username kudu)
I20260806 22:49:48.607218 29940 webserver.cc:460] Webserver started at http://127.28.218.195:35817/ using document root <none> and password file <none>
I20260806 22:49:48.607424 29940 fs_manager.cc:359] Metadata directory not provided
I20260806 22:49:48.607491 29940 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:49:48.607642 29940 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:49:48.608606 29940 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056586788339-29547-0/minicluster-data/ts-2/data/instance:
uuid: "7adc6b29809c41b188c7cbfbe25d263a"
format_stamp: "Formatted at 2026-08-06 22:49:48 on dist-test-slave-1xrh"
server_key: "45e17e6f653b2786d891437b2c39ca08"
server_key_iv: "2f1f3e51efbb973c0397f0d76bc629d8"
server_key_version: "encryptionkey@0"
I20260806 22:49:48.608943 29940 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056586788339-29547-0/minicluster-data/ts-2/wal/instance:
uuid: "7adc6b29809c41b188c7cbfbe25d263a"
format_stamp: "Formatted at 2026-08-06 22:49:48 on dist-test-slave-1xrh"
server_key: "45e17e6f653b2786d891437b2c39ca08"
server_key_iv: "2f1f3e51efbb973c0397f0d76bc629d8"
server_key_version: "encryptionkey@0"
I20260806 22:49:48.610260 29940 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:49:48.610956 29955 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:48.611171 29940 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:49:48.611282 29940 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056586788339-29547-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056586788339-29547-0/minicluster-data/ts-2/wal
uuid: "7adc6b29809c41b188c7cbfbe25d263a"
format_stamp: "Formatted at 2026-08-06 22:49:48 on dist-test-slave-1xrh"
server_key: "45e17e6f653b2786d891437b2c39ca08"
server_key_iv: "2f1f3e51efbb973c0397f0d76bc629d8"
server_key_version: "encryptionkey@0"
I20260806 22:49:48.611356 29940 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056586788339-29547-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056586788339-29547-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056586788339-29547-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:49:48.634274 29940 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:49:48.636168 29940 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:49:48.636286 29940 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:49:48.636463 29940 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:49:48.636713 29940 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:49:48.636742 29940 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:48.636762 29940 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:49:48.636775 29940 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:48.642853 29940 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.195:41857
I20260806 22:49:48.642932 30071 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.195:41857 every 8 connection(s)
I20260806 22:49:48.643273 29940 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056586788339-29547-0/minicluster-data/ts-2/data/info.pb
Aug 06 22:49:48 dist-test-slave-1xrh krb5kdc[29555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056588, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.195@KRBTEST.COM for kudu/127.28.218.254@KRBTEST.COM
I20260806 22:49:48.649725 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 29940
I20260806 22:49:48.649822 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056586788339-29547-0/minicluster-data/ts-2/wal/instance
I20260806 22:49:48.650033 29547 external_mini_cluster.cc:1428] Setting key 6fcb54454f110dacf2bb69510613e022
I20260806 22:49:48.652487 29662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:48.643977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.195:51279 (local address 127.28.218.254:40675)
0806 22:49:48.644041 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:48.644043 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:48.644281 (+ 238us) server_negotiation.cc:407] Connection header received
0806 22:49:48.644802 (+ 521us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:48.644804 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:48.644818 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:48.644864 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:48.645792 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:48.646305 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:48.647336 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:48.647543 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:48.647614 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:48.649600 (+ 1986us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:48.649601 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:48.649602 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:49:48.649611 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:48.651420 (+ 1809us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:48.651765 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:48.651766 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:48.651766 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:48.651814 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:48.652015 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:48.652016 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:48.652016 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:48.652111 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:48.652159 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:48.652369 (+ 210us) server_negotiation.cc:299] Negotiation successful
0806 22:49:48.652425 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 22:49:48.652710 30074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:48.643948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:40675 (local address 127.28.218.195:51279)
0806 22:49:48.644118 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:48.644183 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:48.644691 (+ 508us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:48.644938 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:48.644942 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:48.645502 (+ 560us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:49:48.645693 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:48.645698 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:48.646399 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:48.646400 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:48.647241 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:48.647244 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:48.647632 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:48.647633 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:48.647772 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:48.648171 (+ 399us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:49:48.648179 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:49:48.649507 (+ 1328us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:49:48.651487 (+ 1980us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:48.651492 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:48.651493 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:48.651696 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:48.651873 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:48.651873 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:48.651874 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:48.651963 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:48.652163 (+ 200us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:49:48.652166 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:49:48.652267 (+ 101us) client_negotiation.cc:769] Sending connection context
0806 22:49:48.652342 (+ 75us) client_negotiation.cc:240] Negotiation successful
0806 22:49:48.652464 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":93,"threads_started":1}
I20260806 22:49:48.653110 30072 heartbeater.cc:347] Connected to a master server at 127.28.218.254:40675
I20260806 22:49:48.653190 30072 heartbeater.cc:464] Registering TS with master...
I20260806 22:49:48.653314 30072 heartbeater.cc:511] Master 127.28.218.254:40675 requested a full tablet report, sending...
I20260806 22:49:48.653637 29600 ts_manager.cc:194] Registered new tserver with Master: 7adc6b29809c41b188c7cbfbe25d263a (127.28.218.195:41857)
I20260806 22:49:48.654019 29600 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.218.195@KRBTEST.COM'} at 127.28.218.195:51279
I20260806 22:49:48.662048 29547 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 22:49:48 dist-test-slave-1xrh krb5kdc[29555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056588, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 22:49:48 dist-test-slave-1xrh krb5kdc[29555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056588, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.28.218.193@KRBTEST.COM
I20260806 22:49:48.685223 30079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:48.677783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56602 (local address 127.28.218.193:34039)
0806 22:49:48.678001 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:48.678005 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:48.678017 (+ 12us) server_negotiation.cc:407] Connection header received
0806 22:49:48.678042 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:48.678043 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:48.678057 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:48.678124 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:48.678644 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:48.679443 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:48.680133 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:48.680426 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:48.680480 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:48.682580 (+ 2100us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:48.682588 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:48.682588 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:49:48.682602 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:48.684037 (+ 1435us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:48.684384 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:48.684387 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:48.684387 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:48.684447 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:48.684648 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:48.684649 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:48.684650 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:48.684782 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:48.684924 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:48.685020 (+ 96us) server_negotiation.cc:299] Negotiation successful
0806 22:49:48.685100 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":141,"threads_started":1}
W20260806 22:49:48.685582 29709 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56602
Aug 06 22:49:48 dist-test-slave-1xrh krb5kdc[29555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056588, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 22:49:48 dist-test-slave-1xrh krb5kdc[29555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056588, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.218.193@KRBTEST.COM
I20260806 22:49:48.707635 30079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:48.700887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56610 (local address 127.28.218.193:34039)
0806 22:49:48.700946 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:48.700949 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:48.701075 (+ 126us) server_negotiation.cc:407] Connection header received
0806 22:49:48.701123 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:48.701124 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:48.701134 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:48.701182 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:48.701687 (+ 505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:48.702156 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:48.702833 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:48.703016 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:48.703063 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:48.705458 (+ 2395us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:48.705459 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:48.705459 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:49:48.705471 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:48.706609 (+ 1138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:48.706934 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:48.706935 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:48.706935 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:48.706983 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:48.707215 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:48.707215 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:48.707216 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:48.707302 (+ 86us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:48.707388 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:48.707484 (+ 96us) server_negotiation.cc:299] Negotiation successful
0806 22:49:48.707540 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 22:49:48.708130 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 29666
I20260806 22:49:48.715583 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 29803
I20260806 22:49:48.720772 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 29940
I20260806 22:49:48.725440 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 29568
2026-08-06T22:49:48Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (1945 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056586788339-29547-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:49:48 dist-test-slave-1xrh krb5kdc[30086](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:49:48 dist-test-slave-1xrh krb5kdc[30086](info): set up 2 sockets
Aug 06 22:49:48 dist-test-slave-1xrh krb5kdc[30086](info): commencing operation
krb5kdc: starting...
W20260806 22:49:49.781901 29547 mini_kdc.cc:121] Time spent starting KDC: real 1.036s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056586788339-29547-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056586788339-29547-0/krb5kdc/test-user.keytab.
Aug 06 22:49:49 dist-test-slave-1xrh krb5kdc[30086](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056589, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:49:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:49:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.218.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.218.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.218.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab.
Entry for principal kudu/127.28.218.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab.
I20260806 22:49:49.901785 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056586788339-29547-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056586788339-29547-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056586788339-29547-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.218.254:35767
--webserver_interface=127.28.218.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab
--principal=kudu/127.28.218.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.218.212:37045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.218.254:35767
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:49:49.981761 30100 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:49:49.982002 30100 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:49:49.982038 30100 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:49:49.983191 30100 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:49:49.983260 30100 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:49:49.983284 30100 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:49:49.983305 30100 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:49:49.983335 30100 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:49:49.984805 30100 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:37045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056586788339-29547-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056586788339-29547-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.218.254:35767
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab
--principal=kudu/127.28.218.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.218.254:35767
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.218.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30100
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056586788339-29547-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:49:49.985080 30100 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:49:49.985302 30100 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:49:49.990119 30106 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:49.990093 30105 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:49.990103 30108 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:49:49.990263 30100 server_base.cc:1034] running on GCE node
I20260806 22:49:49.990523 30100 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:49:49.990797 30100 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:49:49.991952 30100 hybrid_clock.cc:648] HybridClock initialized: now 1786056589991937 us; error 49 us; skew 500 ppm
Aug 06 22:49:49 dist-test-slave-1xrh krb5kdc[30086](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056589, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:49:49.999588 30100 init.cc:377] Logged in from keytab as kudu/127.28.218.254@KRBTEST.COM (short username kudu)
I20260806 22:49:50.000018 30100 webserver.cc:460] Webserver started at http://127.28.218.254:45121/ using document root <none> and password file <none>
I20260806 22:49:50.000257 30100 fs_manager.cc:359] Metadata directory not provided
I20260806 22:49:50.000325 30100 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:49:50.000478 30100 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:49:50.001511 30100 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056586788339-29547-0/minicluster-data/master-0/data/instance:
uuid: "d49801c5c9d1439ca32796742b802c8b"
format_stamp: "Formatted at 2026-08-06 22:49:49 on dist-test-slave-1xrh"
server_key: "4ef1e5d9362aaac55551c066e2db3676"
server_key_iv: "fec39d5d2201c0a3affa7eb2b403e9fd"
server_key_version: "encryptionkey@0"
I20260806 22:49:50.001874 30100 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056586788339-29547-0/minicluster-data/master-0/wal/instance:
uuid: "d49801c5c9d1439ca32796742b802c8b"
format_stamp: "Formatted at 2026-08-06 22:49:49 on dist-test-slave-1xrh"
server_key: "4ef1e5d9362aaac55551c066e2db3676"
server_key_iv: "fec39d5d2201c0a3affa7eb2b403e9fd"
server_key_version: "encryptionkey@0"
I20260806 22:49:50.003253 30100 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 22:49:50.004031 30115 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:50.004221 30100 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:49:50.004321 30100 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056586788339-29547-0/minicluster-data/master-0/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056586788339-29547-0/minicluster-data/master-0/wal
uuid: "d49801c5c9d1439ca32796742b802c8b"
format_stamp: "Formatted at 2026-08-06 22:49:49 on dist-test-slave-1xrh"
server_key: "4ef1e5d9362aaac55551c066e2db3676"
server_key_iv: "fec39d5d2201c0a3affa7eb2b403e9fd"
server_key_version: "encryptionkey@0"
I20260806 22:49:50.004408 30100 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056586788339-29547-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056586788339-29547-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056586788339-29547-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:49:50.012315 30100 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:49:50.014040 30100 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:49:50.014204 30100 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:49:50.018244 30100 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.254:35767
I20260806 22:49:50.018275 30169 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.254:35767 every 8 connection(s)
I20260806 22:49:50.018606 30100 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
I20260806 22:49:50.019496 30170 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:49:50.021677 30170 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d49801c5c9d1439ca32796742b802c8b: Bootstrap starting.
I20260806 22:49:50.022240 30170 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d49801c5c9d1439ca32796742b802c8b: Neither blocks nor log segments found. Creating new log.
I20260806 22:49:50.022533 30170 log.cc:824] T 00000000000000000000000000000000 P d49801c5c9d1439ca32796742b802c8b: Log is configured to *not* fsync() on all Append() calls
I20260806 22:49:50.023269 30170 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d49801c5c9d1439ca32796742b802c8b: No bootstrap required, opened a new log
I20260806 22:49:50.024856 30170 raft_consensus.cc:348] T 00000000000000000000000000000000 P d49801c5c9d1439ca32796742b802c8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d49801c5c9d1439ca32796742b802c8b" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 35767 } }
I20260806 22:49:50.024993 30170 raft_consensus.cc:374] T 00000000000000000000000000000000 P d49801c5c9d1439ca32796742b802c8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:49:50.025017 30170 raft_consensus.cc:729] T 00000000000000000000000000000000 P d49801c5c9d1439ca32796742b802c8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d49801c5c9d1439ca32796742b802c8b, State: Initialized, Role: FOLLOWER
I20260806 22:49:50.025192 30170 consensus_queue.cc:260] T 00000000000000000000000000000000 P d49801c5c9d1439ca32796742b802c8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d49801c5c9d1439ca32796742b802c8b" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 35767 } }
I20260806 22:49:50.025254 30170 raft_consensus.cc:388] T 00000000000000000000000000000000 P d49801c5c9d1439ca32796742b802c8b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:49:50.025274 30170 raft_consensus.cc:482] T 00000000000000000000000000000000 P d49801c5c9d1439ca32796742b802c8b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:49:50.025341 30170 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d49801c5c9d1439ca32796742b802c8b [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:49:50.025849 30170 raft_consensus.cc:504] T 00000000000000000000000000000000 P d49801c5c9d1439ca32796742b802c8b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d49801c5c9d1439ca32796742b802c8b" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 35767 } }
I20260806 22:49:50.025961 30170 leader_election.cc:302] T 00000000000000000000000000000000 P d49801c5c9d1439ca32796742b802c8b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d49801c5c9d1439ca32796742b802c8b; no voters:
I20260806 22:49:50.026140 30170 leader_election.cc:288] T 00000000000000000000000000000000 P d49801c5c9d1439ca32796742b802c8b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:49:50.026167 30173 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d49801c5c9d1439ca32796742b802c8b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:49:50.026445 30170 sys_catalog.cc:564] T 00000000000000000000000000000000 P d49801c5c9d1439ca32796742b802c8b [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:49:50.026477 30173 raft_consensus.cc:686] T 00000000000000000000000000000000 P d49801c5c9d1439ca32796742b802c8b [term 1 LEADER]: Becoming Leader. State: Replica: d49801c5c9d1439ca32796742b802c8b, State: Running, Role: LEADER
I20260806 22:49:50.026566 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 30100
I20260806 22:49:50.026651 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056586788339-29547-0/minicluster-data/master-0/wal/instance
I20260806 22:49:50.026690 30173 consensus_queue.cc:237] T 00000000000000000000000000000000 P d49801c5c9d1439ca32796742b802c8b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d49801c5c9d1439ca32796742b802c8b" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 35767 } }
I20260806 22:49:50.026808 29547 external_mini_cluster.cc:1428] Setting key 64dbcff31c0080ef7f7bea4cc8f11c5c
I20260806 22:49:50.027701 30174 sys_catalog.cc:455] T 00000000000000000000000000000000 P d49801c5c9d1439ca32796742b802c8b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d49801c5c9d1439ca32796742b802c8b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d49801c5c9d1439ca32796742b802c8b" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 35767 } } }
I20260806 22:49:50.027818 30174 sys_catalog.cc:458] T 00000000000000000000000000000000 P d49801c5c9d1439ca32796742b802c8b [sys.catalog]: This master's current role is: LEADER
I20260806 22:49:50.027762 30175 sys_catalog.cc:455] T 00000000000000000000000000000000 P d49801c5c9d1439ca32796742b802c8b [sys.catalog]: SysCatalogTable state changed. Reason: New leader d49801c5c9d1439ca32796742b802c8b. Latest consensus state: current_term: 1 leader_uuid: "d49801c5c9d1439ca32796742b802c8b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d49801c5c9d1439ca32796742b802c8b" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 35767 } } }
I20260806 22:49:50.028008 30175 sys_catalog.cc:458] T 00000000000000000000000000000000 P d49801c5c9d1439ca32796742b802c8b [sys.catalog]: This master's current role is: LEADER
W20260806 22:49:50.028468 30192 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d49801c5c9d1439ca32796742b802c8b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 22:49:50.028529 30192 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 22:49:50.028582 30191 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:49:50.028744 30191 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:49:50.030110 30191 catalog_manager.cc:1347] Generated new cluster ID: 0a642a6759834c82a97f40fd30ac505c
I20260806 22:49:50.030164 30191 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 22:49:50 dist-test-slave-1xrh krb5kdc[30086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056589, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.218.254@KRBTEST.COM
I20260806 22:49:50.037892 30183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:50.027326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51252 (local address 127.28.218.254:35767)
0806 22:49:50.028271 (+ 945us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:50.028281 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:50.028314 (+ 33us) server_negotiation.cc:407] Connection header received
0806 22:49:50.028421 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:50.028426 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:50.028469 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:50.028612 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:50.029348 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:50.030469 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:50.031291 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:50.031701 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:50.031818 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:50.034082 (+ 2264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:50.034086 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:50.034086 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:49:50.034099 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:50.036353 (+ 2254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:50.036726 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:50.036729 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:50.036729 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:50.036830 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:50.037043 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:50.037044 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:50.037044 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:50.037279 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:50.037449 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:50.037542 (+ 93us) server_negotiation.cc:299] Negotiation successful
0806 22:49:50.037682 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":899,"thread_start_us":91,"threads_started":1}
I20260806 22:49:50.047868 30191 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:49:50.048358 30191 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:49:50.053520 30191 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d49801c5c9d1439ca32796742b802c8b: Generated new TSK 0
I20260806 22:49:50.053691 30191 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.28.218.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.218.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.218.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.193.keytab.
Entry for principal kudu/127.28.218.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.193.keytab.
I20260806 22:49:50.082170 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056586788339-29547-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056586788339-29547-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056586788339-29547-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056586788339-29547-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.218.193:0
--local_ip_for_outbound_sockets=127.28.218.193
--webserver_interface=127.28.218.193
--webserver_port=0
--tserver_master_addrs=127.28.218.254:35767
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.193.keytab
--principal=kudu/127.28.218.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.218.212:37045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:49:50.163096 30197 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:49:50.163327 30197 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:49:50.163365 30197 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:49:50.164532 30197 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:49:50.164598 30197 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:49:50.164644 30197 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.218.193
I20260806 22:49:50.166106 30197 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:37045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056586788339-29547-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056586788339-29547-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.193.keytab
--principal=kudu/127.28.218.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.218.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056586788339-29547-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.218.193
--webserver_port=0
--tserver_master_addrs=127.28.218.254:35767
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30197
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.218.193
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056586788339-29547-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:49:50.166363 30197 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:49:50.166584 30197 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:49:50.171358 30202 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:50.171365 30203 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:49:50.171500 30197 server_base.cc:1034] running on GCE node
W20260806 22:49:50.171362 30205 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:49:50.171726 30197 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:49:50.171941 30197 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:49:50.173081 30197 hybrid_clock.cc:648] HybridClock initialized: now 1786056590173074 us; error 30 us; skew 500 ppm
Aug 06 22:49:50 dist-test-slave-1xrh krb5kdc[30086](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056590, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:49:50.175544 30197 init.cc:377] Logged in from keytab as kudu/127.28.218.193@KRBTEST.COM (short username kudu)
I20260806 22:49:50.175941 30197 webserver.cc:460] Webserver started at http://127.28.218.193:44001/ using document root <none> and password file <none>
I20260806 22:49:50.176172 30197 fs_manager.cc:359] Metadata directory not provided
I20260806 22:49:50.176216 30197 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:49:50.176378 30197 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:49:50.177342 30197 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056586788339-29547-0/minicluster-data/ts-0/data/instance:
uuid: "b0093d82fdb548d1b85c099e805a29b3"
format_stamp: "Formatted at 2026-08-06 22:49:50 on dist-test-slave-1xrh"
server_key: "8d8664a40ac87011d4bccd42f0c7eef6"
server_key_iv: "14080898c6b8ee3987b7724e8b8e7df5"
server_key_version: "encryptionkey@0"
I20260806 22:49:50.177691 30197 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056586788339-29547-0/minicluster-data/ts-0/wal/instance:
uuid: "b0093d82fdb548d1b85c099e805a29b3"
format_stamp: "Formatted at 2026-08-06 22:49:50 on dist-test-slave-1xrh"
server_key: "8d8664a40ac87011d4bccd42f0c7eef6"
server_key_iv: "14080898c6b8ee3987b7724e8b8e7df5"
server_key_version: "encryptionkey@0"
I20260806 22:49:50.178958 30197 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 22:49:50.179730 30212 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:50.179898 30197 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:49:50.179986 30197 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056586788339-29547-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056586788339-29547-0/minicluster-data/ts-0/wal
uuid: "b0093d82fdb548d1b85c099e805a29b3"
format_stamp: "Formatted at 2026-08-06 22:49:50 on dist-test-slave-1xrh"
server_key: "8d8664a40ac87011d4bccd42f0c7eef6"
server_key_iv: "14080898c6b8ee3987b7724e8b8e7df5"
server_key_version: "encryptionkey@0"
I20260806 22:49:50.180069 30197 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056586788339-29547-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056586788339-29547-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056586788339-29547-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:49:50.198405 30197 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:49:50.200291 30197 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:49:50.200448 30197 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:49:50.200691 30197 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:49:50.200985 30197 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:49:50.201038 30197 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:50.201078 30197 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:49:50.201107 30197 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:50.206811 30197 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.193:46511
I20260806 22:49:50.206852 30327 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.193:46511 every 8 connection(s)
I20260806 22:49:50.207268 30197 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056586788339-29547-0/minicluster-data/ts-0/data/info.pb
Aug 06 22:49:50 dist-test-slave-1xrh krb5kdc[30086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056590, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.193@KRBTEST.COM for kudu/127.28.218.254@KRBTEST.COM
I20260806 22:49:50.216164 30183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:50.207985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.193:32863 (local address 127.28.218.254:35767)
0806 22:49:50.208078 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:50.208080 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:50.208185 (+ 105us) server_negotiation.cc:407] Connection header received
0806 22:49:50.209264 (+ 1079us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:50.209265 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:50.209275 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:50.209327 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:50.209882 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:50.210371 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:50.211361 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:50.211548 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:50.211602 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:50.213602 (+ 2000us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:50.213604 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:50.213604 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:49:50.213610 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:50.215010 (+ 1400us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:50.215416 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:50.215417 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:50.215417 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:50.215471 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:50.215677 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:50.215677 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:50.215677 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:50.215774 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:50.215824 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:50.216041 (+ 217us) server_negotiation.cc:299] Negotiation successful
0806 22:49:50.216102 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 22:49:50.216318 30330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:50.207928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:35767 (local address 127.28.218.193:32863)
0806 22:49:50.208088 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:50.208123 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:50.209171 (+ 1048us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:50.209378 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:50.209382 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:50.209629 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:49:50.209812 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:50.209816 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:50.210456 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:50.210456 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:50.211291 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:50.211293 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:50.211621 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:50.211622 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:50.211714 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:50.212139 (+ 425us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:49:50.212147 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:49:50.213520 (+ 1373us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:49:50.215075 (+ 1555us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:50.215080 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:50.215082 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:50.215353 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:50.215531 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:50.215531 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:50.215531 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:50.215624 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:50.215829 (+ 205us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:49:50.215833 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:49:50.215942 (+ 109us) client_negotiation.cc:769] Sending connection context
0806 22:49:50.216022 (+ 80us) client_negotiation.cc:240] Negotiation successful
0806 22:49:50.216131 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":93,"threads_started":1}
I20260806 22:49:50.216781 30328 heartbeater.cc:347] Connected to a master server at 127.28.218.254:35767
I20260806 22:49:50.216857 30328 heartbeater.cc:464] Registering TS with master...
I20260806 22:49:50.217006 30328 heartbeater.cc:511] Master 127.28.218.254:35767 requested a full tablet report, sending...
I20260806 22:49:50.217262 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 30197
I20260806 22:49:50.217360 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056586788339-29547-0/minicluster-data/ts-0/wal/instance
I20260806 22:49:50.217535 29547 external_mini_cluster.cc:1428] Setting key a7ac4e8e20e25a3bfe96e768daedc4dc
I20260806 22:49:50.217537 30133 ts_manager.cc:194] Registered new tserver with Master: b0093d82fdb548d1b85c099e805a29b3 (127.28.218.193:46511)
I20260806 22:49:50.218218 30133 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.218.193@KRBTEST.COM'} at 127.28.218.193:32863
WARNING: no policy specified for kudu/127.28.218.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.218.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.218.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.194.keytab.
Entry for principal kudu/127.28.218.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.194.keytab.
I20260806 22:49:50.244179 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056586788339-29547-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056586788339-29547-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056586788339-29547-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056586788339-29547-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.218.194:0
--local_ip_for_outbound_sockets=127.28.218.194
--webserver_interface=127.28.218.194
--webserver_port=0
--tserver_master_addrs=127.28.218.254:35767
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.194.keytab
--principal=kudu/127.28.218.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.218.212:37045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:49:50.324749 30333 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:49:50.324954 30333 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:49:50.324977 30333 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:49:50.326104 30333 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:49:50.326150 30333 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:49:50.326176 30333 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.218.194
I20260806 22:49:50.328018 30333 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:37045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056586788339-29547-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056586788339-29547-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.194.keytab
--principal=kudu/127.28.218.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.218.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056586788339-29547-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.218.194
--webserver_port=0
--tserver_master_addrs=127.28.218.254:35767
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30333
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.218.194
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056586788339-29547-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:49:50.328241 30333 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:49:50.328425 30333 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:49:50.333268 30339 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:50.333314 30338 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:49:50.333518 30333 server_base.cc:1034] running on GCE node
W20260806 22:49:50.333314 30341 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:49:50.333799 30333 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:49:50.333998 30333 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:49:50.335202 30333 hybrid_clock.cc:648] HybridClock initialized: now 1786056590335169 us; error 38 us; skew 500 ppm
Aug 06 22:49:50 dist-test-slave-1xrh krb5kdc[30086](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056590, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:49:50.337781 30333 init.cc:377] Logged in from keytab as kudu/127.28.218.194@KRBTEST.COM (short username kudu)
I20260806 22:49:50.338150 30333 webserver.cc:460] Webserver started at http://127.28.218.194:35111/ using document root <none> and password file <none>
I20260806 22:49:50.338333 30333 fs_manager.cc:359] Metadata directory not provided
I20260806 22:49:50.338371 30333 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:49:50.338456 30333 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:49:50.339546 30333 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056586788339-29547-0/minicluster-data/ts-1/data/instance:
uuid: "2517613277294e38968080b309891a73"
format_stamp: "Formatted at 2026-08-06 22:49:50 on dist-test-slave-1xrh"
server_key: "4331f6b9cc4a65f5b6649655081d2915"
server_key_iv: "e1133e97cd1cd98ba7db6120212cfaf7"
server_key_version: "encryptionkey@0"
I20260806 22:49:50.339850 30333 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056586788339-29547-0/minicluster-data/ts-1/wal/instance:
uuid: "2517613277294e38968080b309891a73"
format_stamp: "Formatted at 2026-08-06 22:49:50 on dist-test-slave-1xrh"
server_key: "4331f6b9cc4a65f5b6649655081d2915"
server_key_iv: "e1133e97cd1cd98ba7db6120212cfaf7"
server_key_version: "encryptionkey@0"
I20260806 22:49:50.341059 30333 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:49:50.341706 30348 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:50.341873 30333 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:49:50.341955 30333 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056586788339-29547-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056586788339-29547-0/minicluster-data/ts-1/wal
uuid: "2517613277294e38968080b309891a73"
format_stamp: "Formatted at 2026-08-06 22:49:50 on dist-test-slave-1xrh"
server_key: "4331f6b9cc4a65f5b6649655081d2915"
server_key_iv: "e1133e97cd1cd98ba7db6120212cfaf7"
server_key_version: "encryptionkey@0"
I20260806 22:49:50.342007 30333 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056586788339-29547-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056586788339-29547-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056586788339-29547-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:49:50.358028 30333 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:49:50.360191 30333 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:49:50.360299 30333 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:49:50.360479 30333 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:49:50.360780 30333 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:49:50.360813 30333 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:50.360836 30333 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:49:50.360886 30333 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:50.366374 30333 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.194:41113
I20260806 22:49:50.366448 30466 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.194:41113 every 8 connection(s)
I20260806 22:49:50.366745 30333 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056586788339-29547-0/minicluster-data/ts-1/data/info.pb
I20260806 22:49:50.368752 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 30333
I20260806 22:49:50.368851 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056586788339-29547-0/minicluster-data/ts-1/wal/instance
I20260806 22:49:50.369031 29547 external_mini_cluster.cc:1428] Setting key 691bdc93e6604fdf9c4ebc7f2237033f
Aug 06 22:49:50 dist-test-slave-1xrh krb5kdc[30086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056590, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.194@KRBTEST.COM for kudu/127.28.218.254@KRBTEST.COM
I20260806 22:49:50.377149 30183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:50.367492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.194:41717 (local address 127.28.218.254:35767)
0806 22:49:50.367577 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:50.367579 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:50.367752 (+ 173us) server_negotiation.cc:407] Connection header received
0806 22:49:50.368682 (+ 930us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:50.368683 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:50.368695 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:50.368747 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:50.369501 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:50.370201 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:50.371187 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:50.371479 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:50.371539 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:50.373915 (+ 2376us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:50.373917 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:50.373917 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:49:50.373926 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:50.375622 (+ 1696us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:50.376029 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:50.376031 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:50.376031 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:50.376095 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:50.376374 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:50.376374 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:50.376375 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:50.376518 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:50.376595 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:50.376820 (+ 225us) server_negotiation.cc:299] Negotiation successful
0806 22:49:50.376912 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 22:49:50.377234 30469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:50.367437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:35767 (local address 127.28.218.194:41717)
0806 22:49:50.367626 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:50.367670 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:50.368574 (+ 904us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:50.368813 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:50.368817 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:50.369156 (+ 339us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:49:50.369414 (+ 258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:50.369420 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:50.370293 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:50.370294 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:50.371052 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:50.371053 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:50.371573 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:50.371575 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:50.371721 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:50.372180 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:49:50.372190 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:49:50.373805 (+ 1615us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:49:50.375715 (+ 1910us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:50.375721 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:50.375723 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:50.375956 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:50.376205 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:50.376205 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:50.376206 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:50.376308 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:50.376587 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:49:50.376591 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:49:50.376696 (+ 105us) client_negotiation.cc:769] Sending connection context
0806 22:49:50.376773 (+ 77us) client_negotiation.cc:240] Negotiation successful
0806 22:49:50.376892 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":87,"threads_started":1}
I20260806 22:49:50.377759 30467 heartbeater.cc:347] Connected to a master server at 127.28.218.254:35767
I20260806 22:49:50.377871 30467 heartbeater.cc:464] Registering TS with master...
I20260806 22:49:50.378043 30467 heartbeater.cc:511] Master 127.28.218.254:35767 requested a full tablet report, sending...
I20260806 22:49:50.378472 30133 ts_manager.cc:194] Registered new tserver with Master: 2517613277294e38968080b309891a73 (127.28.218.194:41113)
WARNING: no policy specified for kudu/127.28.218.195@KRBTEST.COM; defaulting to no policy
I20260806 22:49:50.379011 30133 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.218.194@KRBTEST.COM'} at 127.28.218.194:41717
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.218.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.218.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.195.keytab.
Entry for principal kudu/127.28.218.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.195.keytab.
I20260806 22:49:50.397641 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056586788339-29547-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056586788339-29547-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056586788339-29547-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056586788339-29547-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.218.195:0
--local_ip_for_outbound_sockets=127.28.218.195
--webserver_interface=127.28.218.195
--webserver_port=0
--tserver_master_addrs=127.28.218.254:35767
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.195.keytab
--principal=kudu/127.28.218.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.218.212:37045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:49:50.477517 30472 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:49:50.477723 30472 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:49:50.477746 30472 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:49:50.478880 30472 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:49:50.478921 30472 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:49:50.478981 30472 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.218.195
I20260806 22:49:50.480458 30472 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:37045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056586788339-29547-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056586788339-29547-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.195.keytab
--principal=kudu/127.28.218.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.218.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056586788339-29547-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.218.195
--webserver_port=0
--tserver_master_addrs=127.28.218.254:35767
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30472
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.218.195
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056586788339-29547-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:49:50.480690 30472 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:49:50.480911 30472 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:49:50.485841 30477 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:50.485863 30478 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:50.485893 30481 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:49:50.485956 30472 server_base.cc:1034] running on GCE node
I20260806 22:49:50.486266 30472 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:49:50.486510 30472 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:49:50.487663 30472 hybrid_clock.cc:648] HybridClock initialized: now 1786056590487640 us; error 52 us; skew 500 ppm
Aug 06 22:49:50 dist-test-slave-1xrh krb5kdc[30086](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056590, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:49:50.490010 30472 init.cc:377] Logged in from keytab as kudu/127.28.218.195@KRBTEST.COM (short username kudu)
I20260806 22:49:50.490442 30472 webserver.cc:460] Webserver started at http://127.28.218.195:40293/ using document root <none> and password file <none>
I20260806 22:49:50.490677 30472 fs_manager.cc:359] Metadata directory not provided
I20260806 22:49:50.490743 30472 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:49:50.490892 30472 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:49:50.491895 30472 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056586788339-29547-0/minicluster-data/ts-2/data/instance:
uuid: "670d15cb22af48bba05f7ce600523650"
format_stamp: "Formatted at 2026-08-06 22:49:50 on dist-test-slave-1xrh"
server_key: "b91afde8660d100f767d855a8d2714f6"
server_key_iv: "163ce00b45145d7f0bf148641cf01a98"
server_key_version: "encryptionkey@0"
I20260806 22:49:50.492233 30472 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056586788339-29547-0/minicluster-data/ts-2/wal/instance:
uuid: "670d15cb22af48bba05f7ce600523650"
format_stamp: "Formatted at 2026-08-06 22:49:50 on dist-test-slave-1xrh"
server_key: "b91afde8660d100f767d855a8d2714f6"
server_key_iv: "163ce00b45145d7f0bf148641cf01a98"
server_key_version: "encryptionkey@0"
I20260806 22:49:50.493484 30472 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 22:49:50.494150 30488 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:50.494303 30472 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:49:50.494418 30472 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056586788339-29547-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056586788339-29547-0/minicluster-data/ts-2/wal
uuid: "670d15cb22af48bba05f7ce600523650"
format_stamp: "Formatted at 2026-08-06 22:49:50 on dist-test-slave-1xrh"
server_key: "b91afde8660d100f767d855a8d2714f6"
server_key_iv: "163ce00b45145d7f0bf148641cf01a98"
server_key_version: "encryptionkey@0"
I20260806 22:49:50.494513 30472 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056586788339-29547-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056586788339-29547-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056586788339-29547-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:49:50.509744 30472 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:49:50.511494 30472 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:49:50.511663 30472 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:49:50.511878 30472 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:49:50.512203 30472 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:49:50.512255 30472 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:50.512295 30472 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:49:50.512321 30472 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:50.518392 30472 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.195:37555
I20260806 22:49:50.518424 30606 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.195:37555 every 8 connection(s)
I20260806 22:49:50.518759 30472 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056586788339-29547-0/minicluster-data/ts-2/data/info.pb
I20260806 22:49:50.522714 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 30472
I20260806 22:49:50.522817 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056586788339-29547-0/minicluster-data/ts-2/wal/instance
I20260806 22:49:50.523018 29547 external_mini_cluster.cc:1428] Setting key 9330d7c24c273a255c57af70a70d3edc
Aug 06 22:49:50 dist-test-slave-1xrh krb5kdc[30086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056590, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.195@KRBTEST.COM for kudu/127.28.218.254@KRBTEST.COM
I20260806 22:49:50.528576 30183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:50.519566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.195:52517 (local address 127.28.218.254:35767)
0806 22:49:50.519615 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:50.519617 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:50.519908 (+ 291us) server_negotiation.cc:407] Connection header received
0806 22:49:50.520726 (+ 818us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:50.520727 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:50.520736 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:50.520794 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:50.521373 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:50.521845 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:50.523033 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:50.523396 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:50.523472 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:50.525733 (+ 2261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:50.525734 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:50.525735 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:49:50.525741 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:50.526964 (+ 1223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:50.527506 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:50.527507 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:50.527507 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:50.527557 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:50.527801 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:50.527802 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:50.527802 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:50.527905 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:50.528021 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:50.528449 (+ 428us) server_negotiation.cc:299] Negotiation successful
0806 22:49:50.528514 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 22:49:50.528714 30609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:50.519548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:35767 (local address 127.28.218.195:52517)
0806 22:49:50.519714 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:50.519747 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:50.520630 (+ 883us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:50.520865 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:50.520868 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:50.521117 (+ 249us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:49:50.521299 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:50.521303 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:50.521924 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:50.521924 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:50.522898 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:50.522901 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:50.523527 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:50.523528 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:50.523713 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:50.524235 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:49:50.524243 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:49:50.525650 (+ 1407us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:49:50.527056 (+ 1406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:50.527063 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:50.527065 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:50.527421 (+ 356us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:50.527637 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:50.527638 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:50.527638 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:50.527743 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:50.528086 (+ 343us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:49:50.528094 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:49:50.528294 (+ 200us) client_negotiation.cc:769] Sending connection context
0806 22:49:50.528406 (+ 112us) client_negotiation.cc:240] Negotiation successful
0806 22:49:50.528530 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":92,"threads_started":1}
I20260806 22:49:50.529204 30607 heartbeater.cc:347] Connected to a master server at 127.28.218.254:35767
I20260806 22:49:50.529280 30607 heartbeater.cc:464] Registering TS with master...
I20260806 22:49:50.529467 30607 heartbeater.cc:511] Master 127.28.218.254:35767 requested a full tablet report, sending...
I20260806 22:49:50.529853 30133 ts_manager.cc:194] Registered new tserver with Master: 670d15cb22af48bba05f7ce600523650 (127.28.218.195:37555)
I20260806 22:49:50.530241 30133 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.218.195@KRBTEST.COM'} at 127.28.218.195:52517
I20260806 22:49:50.534632 29547 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:49:50.542129 30183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:50.536201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51262 (local address 127.28.218.254:35767)
0806 22:49:50.536277 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:50.536279 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:50.536370 (+ 91us) server_negotiation.cc:407] Connection header received
0806 22:49:50.536476 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:50.536477 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:50.536486 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:50.536557 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:50.537104 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:50.537562 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:50.538410 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:50.538605 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:50.538678 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:50.539852 (+ 1174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:50.539853 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:50.539854 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:49:50.539860 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:50.540985 (+ 1125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:50.541340 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:50.541342 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:50.541342 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:50.541398 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:50.541603 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:50.541604 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:50.541604 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:50.541744 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:50.541849 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:50.541953 (+ 104us) server_negotiation.cc:299] Negotiation successful
0806 22:49:50.542031 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 22:49:50.544184 30133 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51262:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:49:50.545845 30133 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:49:50.551784 30615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:50.547999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.194:41113 (local address 127.0.0.1:39960)
0806 22:49:50.548097 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:50.548132 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:50.548194 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:50.548426 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:50.548429 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:50.548430 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:49:50.548499 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:50.548501 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:50.549483 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:50.549484 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:50.550563 (+ 1079us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:50.550565 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:50.551464 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:50.551465 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:50.551554 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:50.551567 (+ 13us) client_negotiation.cc:769] Sending connection context
0806 22:49:50.551666 (+ 99us) client_negotiation.cc:240] Negotiation successful
0806 22:49:50.551718 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":76,"thread_start_us":55,"threads_started":1}
I20260806 22:49:50.551796 30617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:50.548079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39960 (local address 127.28.218.194:41113)
0806 22:49:50.548231 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:50.548236 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:50.548246 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:49:50.548285 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:50.548287 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:50.548307 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:50.548423 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:49:50.548589 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:50.549378 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:50.550650 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:50.551378 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:50.551465 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:50.551518 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:50.551683 (+ 165us) server_negotiation.cc:299] Negotiation successful
0806 22:49:50.551720 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":79,"threads_started":1}
I20260806 22:49:50.552040 30616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:50.548023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.195:37555 (local address 127.0.0.1:45498)
0806 22:49:50.548285 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:50.548294 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:50.548332 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:50.548656 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:50.548657 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:50.548659 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:49:50.548700 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:50.548702 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:50.549891 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:50.549891 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:50.550833 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:50.550835 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:50.551805 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:50.551806 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:50.551867 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:50.551871 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:49:50.551925 (+ 54us) client_negotiation.cc:240] Negotiation successful
0806 22:49:50.551966 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":77,"threads_started":1}
I20260806 22:49:50.552256 30618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:50.548200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.193:46511 (local address 127.0.0.1:42236)
0806 22:49:50.548588 (+ 388us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:50.548598 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:50.548647 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:50.548853 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:50.548854 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:50.548856 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:49:50.548893 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:50.548894 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:50.550209 (+ 1315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:50.550210 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:50.551115 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:50.551118 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:50.552044 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:50.552045 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:50.552103 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:50.552108 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:49:50.552161 (+ 53us) client_negotiation.cc:240] Negotiation successful
0806 22:49:50.552199 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":44,"threads_started":1}
I20260806 22:49:50.552402 30619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:50.548277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45498 (local address 127.28.218.195:37555)
0806 22:49:50.548492 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:50.548496 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:50.548506 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:49:50.548543 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:50.548545 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:50.548561 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:50.548645 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:49:50.548759 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:50.549780 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:50.550947 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:50.551653 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:50.551724 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:50.551769 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:50.552207 (+ 438us) server_negotiation.cc:299] Negotiation successful
0806 22:49:50.552248 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":79,"threads_started":1}
I20260806 22:49:50.552479 30620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:50.548395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42236 (local address 127.28.218.193:46511)
0806 22:49:50.548592 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:50.548596 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:50.548707 (+ 111us) server_negotiation.cc:407] Connection header received
0806 22:49:50.548749 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:50.548751 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:50.548766 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:50.548842 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:49:50.548953 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:50.550119 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:50.551224 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:50.551893 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:50.551953 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:50.552012 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:50.552356 (+ 344us) server_negotiation.cc:299] Negotiation successful
0806 22:49:50.552400 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":90,"threads_started":1}
I20260806 22:49:50.552781 30399 tablet_service.cc:1467] Processing CreateTablet for tablet e0fc2ccb0381435b9c4e831157639f90 (DEFAULT_TABLE table=test-table [id=ba9cf8e20bb641018e3ed22925420cc4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:49:50.553146 30399 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e0fc2ccb0381435b9c4e831157639f90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:49:50.553126 30262 tablet_service.cc:1467] Processing CreateTablet for tablet e0fc2ccb0381435b9c4e831157639f90 (DEFAULT_TABLE table=test-table [id=ba9cf8e20bb641018e3ed22925420cc4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:49:50.553457 30262 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e0fc2ccb0381435b9c4e831157639f90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:49:50.555743 30538 tablet_service.cc:1467] Processing CreateTablet for tablet e0fc2ccb0381435b9c4e831157639f90 (DEFAULT_TABLE table=test-table [id=ba9cf8e20bb641018e3ed22925420cc4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:49:50.556097 30538 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e0fc2ccb0381435b9c4e831157639f90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:49:50.556624 30622 tablet_bootstrap.cc:492] T e0fc2ccb0381435b9c4e831157639f90 P b0093d82fdb548d1b85c099e805a29b3: Bootstrap starting.
I20260806 22:49:50.556865 30621 tablet_bootstrap.cc:492] T e0fc2ccb0381435b9c4e831157639f90 P 2517613277294e38968080b309891a73: Bootstrap starting.
I20260806 22:49:50.557327 30622 tablet_bootstrap.cc:654] T e0fc2ccb0381435b9c4e831157639f90 P b0093d82fdb548d1b85c099e805a29b3: Neither blocks nor log segments found. Creating new log.
I20260806 22:49:50.557466 30621 tablet_bootstrap.cc:654] T e0fc2ccb0381435b9c4e831157639f90 P 2517613277294e38968080b309891a73: Neither blocks nor log segments found. Creating new log.
I20260806 22:49:50.557899 30622 log.cc:824] T e0fc2ccb0381435b9c4e831157639f90 P b0093d82fdb548d1b85c099e805a29b3: Log is configured to *not* fsync() on all Append() calls
I20260806 22:49:50.557936 30621 log.cc:824] T e0fc2ccb0381435b9c4e831157639f90 P 2517613277294e38968080b309891a73: Log is configured to *not* fsync() on all Append() calls
I20260806 22:49:50.558698 30622 tablet_bootstrap.cc:492] T e0fc2ccb0381435b9c4e831157639f90 P b0093d82fdb548d1b85c099e805a29b3: No bootstrap required, opened a new log
I20260806 22:49:50.558698 30621 tablet_bootstrap.cc:492] T e0fc2ccb0381435b9c4e831157639f90 P 2517613277294e38968080b309891a73: No bootstrap required, opened a new log
I20260806 22:49:50.558770 30622 ts_tablet_manager.cc:1397] T e0fc2ccb0381435b9c4e831157639f90 P b0093d82fdb548d1b85c099e805a29b3: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:49:50.558770 30621 ts_tablet_manager.cc:1397] T e0fc2ccb0381435b9c4e831157639f90 P 2517613277294e38968080b309891a73: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:49:50.559258 30625 tablet_bootstrap.cc:492] T e0fc2ccb0381435b9c4e831157639f90 P 670d15cb22af48bba05f7ce600523650: Bootstrap starting.
I20260806 22:49:50.560009 30625 tablet_bootstrap.cc:654] T e0fc2ccb0381435b9c4e831157639f90 P 670d15cb22af48bba05f7ce600523650: Neither blocks nor log segments found. Creating new log.
I20260806 22:49:50.560498 30625 log.cc:824] T e0fc2ccb0381435b9c4e831157639f90 P 670d15cb22af48bba05f7ce600523650: Log is configured to *not* fsync() on all Append() calls
I20260806 22:49:50.560750 30622 raft_consensus.cc:348] T e0fc2ccb0381435b9c4e831157639f90 P b0093d82fdb548d1b85c099e805a29b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2517613277294e38968080b309891a73" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 41113 } } peers { permanent_uuid: "670d15cb22af48bba05f7ce600523650" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 37555 } } peers { permanent_uuid: "b0093d82fdb548d1b85c099e805a29b3" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 46511 } }
I20260806 22:49:50.560750 30621 raft_consensus.cc:348] T e0fc2ccb0381435b9c4e831157639f90 P 2517613277294e38968080b309891a73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2517613277294e38968080b309891a73" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 41113 } } peers { permanent_uuid: "670d15cb22af48bba05f7ce600523650" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 37555 } } peers { permanent_uuid: "b0093d82fdb548d1b85c099e805a29b3" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 46511 } }
I20260806 22:49:50.560940 30621 raft_consensus.cc:374] T e0fc2ccb0381435b9c4e831157639f90 P 2517613277294e38968080b309891a73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:49:50.560936 30622 raft_consensus.cc:374] T e0fc2ccb0381435b9c4e831157639f90 P b0093d82fdb548d1b85c099e805a29b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:49:50.560982 30622 raft_consensus.cc:729] T e0fc2ccb0381435b9c4e831157639f90 P b0093d82fdb548d1b85c099e805a29b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0093d82fdb548d1b85c099e805a29b3, State: Initialized, Role: FOLLOWER
I20260806 22:49:50.560983 30621 raft_consensus.cc:729] T e0fc2ccb0381435b9c4e831157639f90 P 2517613277294e38968080b309891a73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2517613277294e38968080b309891a73, State: Initialized, Role: FOLLOWER
I20260806 22:49:50.561095 30622 consensus_queue.cc:260] T e0fc2ccb0381435b9c4e831157639f90 P b0093d82fdb548d1b85c099e805a29b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2517613277294e38968080b309891a73" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 41113 } } peers { permanent_uuid: "670d15cb22af48bba05f7ce600523650" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 37555 } } peers { permanent_uuid: "b0093d82fdb548d1b85c099e805a29b3" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 46511 } }
I20260806 22:49:50.561108 30621 consensus_queue.cc:260] T e0fc2ccb0381435b9c4e831157639f90 P 2517613277294e38968080b309891a73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2517613277294e38968080b309891a73" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 41113 } } peers { permanent_uuid: "670d15cb22af48bba05f7ce600523650" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 37555 } } peers { permanent_uuid: "b0093d82fdb548d1b85c099e805a29b3" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 46511 } }
I20260806 22:49:50.561237 30625 tablet_bootstrap.cc:492] T e0fc2ccb0381435b9c4e831157639f90 P 670d15cb22af48bba05f7ce600523650: No bootstrap required, opened a new log
I20260806 22:49:50.561303 30625 ts_tablet_manager.cc:1397] T e0fc2ccb0381435b9c4e831157639f90 P 670d15cb22af48bba05f7ce600523650: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:49:50.561376 30621 ts_tablet_manager.cc:1428] T e0fc2ccb0381435b9c4e831157639f90 P 2517613277294e38968080b309891a73: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 22:49:50.561376 30622 ts_tablet_manager.cc:1428] T e0fc2ccb0381435b9c4e831157639f90 P b0093d82fdb548d1b85c099e805a29b3: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 22:49:50.561499 30467 heartbeater.cc:503] Master 127.28.218.254:35767 was elected leader, sending a full tablet report...
I20260806 22:49:50.561795 30328 heartbeater.cc:503] Master 127.28.218.254:35767 was elected leader, sending a full tablet report...
I20260806 22:49:50.563265 30625 raft_consensus.cc:348] T e0fc2ccb0381435b9c4e831157639f90 P 670d15cb22af48bba05f7ce600523650 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2517613277294e38968080b309891a73" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 41113 } } peers { permanent_uuid: "670d15cb22af48bba05f7ce600523650" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 37555 } } peers { permanent_uuid: "b0093d82fdb548d1b85c099e805a29b3" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 46511 } }
I20260806 22:49:50.563426 30625 raft_consensus.cc:374] T e0fc2ccb0381435b9c4e831157639f90 P 670d15cb22af48bba05f7ce600523650 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:49:50.563472 30625 raft_consensus.cc:729] T e0fc2ccb0381435b9c4e831157639f90 P 670d15cb22af48bba05f7ce600523650 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 670d15cb22af48bba05f7ce600523650, State: Initialized, Role: FOLLOWER
I20260806 22:49:50.563599 30625 consensus_queue.cc:260] T e0fc2ccb0381435b9c4e831157639f90 P 670d15cb22af48bba05f7ce600523650 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2517613277294e38968080b309891a73" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 41113 } } peers { permanent_uuid: "670d15cb22af48bba05f7ce600523650" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 37555 } } peers { permanent_uuid: "b0093d82fdb548d1b85c099e805a29b3" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 46511 } }
I20260806 22:49:50.563887 30607 heartbeater.cc:503] Master 127.28.218.254:35767 was elected leader, sending a full tablet report...
I20260806 22:49:50.564136 30625 ts_tablet_manager.cc:1428] T e0fc2ccb0381435b9c4e831157639f90 P 670d15cb22af48bba05f7ce600523650: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
W20260806 22:49:50.617451 30468 tablet.cc:2369] T e0fc2ccb0381435b9c4e831157639f90 P 2517613277294e38968080b309891a73: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:49:50.654654 30628 raft_consensus.cc:482] T e0fc2ccb0381435b9c4e831157639f90 P 2517613277294e38968080b309891a73 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:49:50.654778 30628 raft_consensus.cc:504] T e0fc2ccb0381435b9c4e831157639f90 P 2517613277294e38968080b309891a73 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2517613277294e38968080b309891a73" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 41113 } } peers { permanent_uuid: "670d15cb22af48bba05f7ce600523650" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 37555 } } peers { permanent_uuid: "b0093d82fdb548d1b85c099e805a29b3" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 46511 } }
I20260806 22:49:50.655285 30628 leader_election.cc:288] T e0fc2ccb0381435b9c4e831157639f90 P 2517613277294e38968080b309891a73 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 670d15cb22af48bba05f7ce600523650 (127.28.218.195:37555), b0093d82fdb548d1b85c099e805a29b3 (127.28.218.193:46511)
I20260806 22:49:50.658067 30469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:50.655487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.195:37555 (local address 127.28.218.194:35509)
0806 22:49:50.655559 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:50.655573 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:50.655632 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:50.655803 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:50.655805 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:50.655807 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:49:50.655875 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:50.655877 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:50.656526 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:50.656527 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:50.657243 (+ 716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:50.657245 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:50.657794 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:50.657795 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:50.657877 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:50.657880 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:49:50.657958 (+ 78us) client_negotiation.cc:240] Negotiation successful
0806 22:49:50.657997 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260806 22:49:50.658067 30619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:50.655561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.194:35509 (local address 127.28.218.195:37555)
0806 22:49:50.655637 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:50.655639 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:50.655653 (+ 14us) server_negotiation.cc:407] Connection header received
0806 22:49:50.655686 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:50.655688 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:50.655699 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:50.655758 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:49:50.655976 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:50.656445 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:50.657337 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:50.657712 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:50.657794 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:50.657803 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:50.657971 (+ 168us) server_negotiation.cc:299] Negotiation successful
0806 22:49:50.658010 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 22:49:50.658334 30620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:50.655784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.194:55739 (local address 127.28.218.193:46511)
0806 22:49:50.655827 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:50.655829 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:50.655933 (+ 104us) server_negotiation.cc:407] Connection header received
0806 22:49:50.656012 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:50.656013 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:50.656023 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:50.656096 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:49:50.656269 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:50.656713 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:50.657607 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:50.657949 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:50.658032 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:50.658039 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:50.658199 (+ 160us) server_negotiation.cc:299] Negotiation successful
0806 22:49:50.658246 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 22:49:50.658461 30560 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0fc2ccb0381435b9c4e831157639f90" candidate_uuid: "2517613277294e38968080b309891a73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "670d15cb22af48bba05f7ce600523650" is_pre_election: true
I20260806 22:49:50.658566 30630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:50.655615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.193:46511 (local address 127.28.218.194:55739)
0806 22:49:50.655867 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:50.655875 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:50.655941 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:50.656102 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:50.656103 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:50.656105 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:49:50.656194 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:50.656201 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:50.656799 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:50.656800 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:50.657516 (+ 716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:50.657518 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:50.658033 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:50.658034 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:50.658104 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:50.658109 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:49:50.658453 (+ 344us) client_negotiation.cc:240] Negotiation successful
0806 22:49:50.658499 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":132,"threads_started":1}
I20260806 22:49:50.658684 30560 raft_consensus.cc:2454] T e0fc2ccb0381435b9c4e831157639f90 P 670d15cb22af48bba05f7ce600523650 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2517613277294e38968080b309891a73 in term 0.
I20260806 22:49:50.658922 30353 leader_election.cc:302] T e0fc2ccb0381435b9c4e831157639f90 P 2517613277294e38968080b309891a73 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2517613277294e38968080b309891a73, 670d15cb22af48bba05f7ce600523650; no voters:
I20260806 22:49:50.658914 30282 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0fc2ccb0381435b9c4e831157639f90" candidate_uuid: "2517613277294e38968080b309891a73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0093d82fdb548d1b85c099e805a29b3" is_pre_election: true
I20260806 22:49:50.659053 30282 raft_consensus.cc:2454] T e0fc2ccb0381435b9c4e831157639f90 P b0093d82fdb548d1b85c099e805a29b3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2517613277294e38968080b309891a73 in term 0.
I20260806 22:49:50.659081 30628 raft_consensus.cc:2781] T e0fc2ccb0381435b9c4e831157639f90 P 2517613277294e38968080b309891a73 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:49:50.659155 30628 raft_consensus.cc:482] T e0fc2ccb0381435b9c4e831157639f90 P 2517613277294e38968080b309891a73 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:49:50.659198 30628 raft_consensus.cc:3037] T e0fc2ccb0381435b9c4e831157639f90 P 2517613277294e38968080b309891a73 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:49:50.659891 30628 raft_consensus.cc:504] T e0fc2ccb0381435b9c4e831157639f90 P 2517613277294e38968080b309891a73 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2517613277294e38968080b309891a73" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 41113 } } peers { permanent_uuid: "670d15cb22af48bba05f7ce600523650" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 37555 } } peers { permanent_uuid: "b0093d82fdb548d1b85c099e805a29b3" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 46511 } }
I20260806 22:49:50.660034 30628 leader_election.cc:288] T e0fc2ccb0381435b9c4e831157639f90 P 2517613277294e38968080b309891a73 [CANDIDATE]: Term 1 election: Requested vote from peers 670d15cb22af48bba05f7ce600523650 (127.28.218.195:37555), b0093d82fdb548d1b85c099e805a29b3 (127.28.218.193:46511)
I20260806 22:49:50.660231 30282 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0fc2ccb0381435b9c4e831157639f90" candidate_uuid: "2517613277294e38968080b309891a73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0093d82fdb548d1b85c099e805a29b3"
I20260806 22:49:50.660246 30560 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0fc2ccb0381435b9c4e831157639f90" candidate_uuid: "2517613277294e38968080b309891a73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "670d15cb22af48bba05f7ce600523650"
I20260806 22:49:50.660320 30560 raft_consensus.cc:3037] T e0fc2ccb0381435b9c4e831157639f90 P 670d15cb22af48bba05f7ce600523650 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:49:50.660320 30282 raft_consensus.cc:3037] T e0fc2ccb0381435b9c4e831157639f90 P b0093d82fdb548d1b85c099e805a29b3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:49:50.661212 30282 raft_consensus.cc:2454] T e0fc2ccb0381435b9c4e831157639f90 P b0093d82fdb548d1b85c099e805a29b3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2517613277294e38968080b309891a73 in term 1.
I20260806 22:49:50.661212 30560 raft_consensus.cc:2454] T e0fc2ccb0381435b9c4e831157639f90 P 670d15cb22af48bba05f7ce600523650 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2517613277294e38968080b309891a73 in term 1.
I20260806 22:49:50.661402 30353 leader_election.cc:302] T e0fc2ccb0381435b9c4e831157639f90 P 2517613277294e38968080b309891a73 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2517613277294e38968080b309891a73, 670d15cb22af48bba05f7ce600523650; no voters:
I20260806 22:49:50.661513 30628 raft_consensus.cc:2781] T e0fc2ccb0381435b9c4e831157639f90 P 2517613277294e38968080b309891a73 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:49:50.661662 30628 raft_consensus.cc:686] T e0fc2ccb0381435b9c4e831157639f90 P 2517613277294e38968080b309891a73 [term 1 LEADER]: Becoming Leader. State: Replica: 2517613277294e38968080b309891a73, State: Running, Role: LEADER
I20260806 22:49:50.661731 30628 consensus_queue.cc:237] T e0fc2ccb0381435b9c4e831157639f90 P 2517613277294e38968080b309891a73 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2517613277294e38968080b309891a73" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 41113 } } peers { permanent_uuid: "670d15cb22af48bba05f7ce600523650" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 37555 } } peers { permanent_uuid: "b0093d82fdb548d1b85c099e805a29b3" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 46511 } }
I20260806 22:49:50.662613 30132 catalog_manager.cc:5495] T e0fc2ccb0381435b9c4e831157639f90 P 2517613277294e38968080b309891a73 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2517613277294e38968080b309891a73 (127.28.218.194). New cstate: current_term: 1 leader_uuid: "2517613277294e38968080b309891a73" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2517613277294e38968080b309891a73" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 41113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "670d15cb22af48bba05f7ce600523650" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 37555 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b0093d82fdb548d1b85c099e805a29b3" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 46511 } health_report { overall_health: UNKNOWN } } }
Aug 06 22:49:50 dist-test-slave-1xrh krb5kdc[30086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056589, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.218.193@KRBTEST.COM
I20260806 22:49:50.699935 30620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:50.692277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42238 (local address 127.28.218.193:46511)
0806 22:49:50.692370 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:50.692372 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:50.692483 (+ 111us) server_negotiation.cc:407] Connection header received
0806 22:49:50.692563 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:50.692564 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:50.692576 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:50.692641 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:50.693214 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:50.693676 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:50.694385 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:50.694574 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:50.694659 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:50.697167 (+ 2508us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:50.697178 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:50.697178 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:49:50.697186 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:50.698709 (+ 1523us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:50.699078 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:50.699088 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:50.699088 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:50.699188 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:50.699387 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:50.699388 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:50.699389 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:50.699539 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:50.699703 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:50.699752 (+ 49us) server_negotiation.cc:299] Negotiation successful
0806 22:49:50.699830 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
W20260806 22:49:50.708071 30329 tablet.cc:2369] T e0fc2ccb0381435b9c4e831157639f90 P b0093d82fdb548d1b85c099e805a29b3: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 06 22:49:50 dist-test-slave-1xrh krb5kdc[30086](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056590, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 22:49:50 dist-test-slave-1xrh krb5kdc[30086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056590, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.28.218.193@KRBTEST.COM
I20260806 22:49:50.722856 30620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:50.716556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42246 (local address 127.28.218.193:46511)
0806 22:49:50.716617 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:50.716619 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:50.716654 (+ 35us) server_negotiation.cc:407] Connection header received
0806 22:49:50.716734 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:50.716735 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:50.716743 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:50.716790 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:50.717284 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:50.717741 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:50.718417 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:50.718590 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:50.718637 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:50.720631 (+ 1994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:50.720632 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:50.720632 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:49:50.720638 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:50.722026 (+ 1388us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:50.722299 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:50.722300 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:50.722300 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:50.722347 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:50.722492 (+ 145us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:50.722493 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:50.722493 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:50.722567 (+ 74us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:50.722630 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:50.722744 (+ 114us) server_negotiation.cc:299] Negotiation successful
0806 22:49:50.722797 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
Aug 06 22:49:50 dist-test-slave-1xrh krb5kdc[30086](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056590, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 22:49:50 dist-test-slave-1xrh krb5kdc[30086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056590, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.218.193@KRBTEST.COM
I20260806 22:49:50.744921 30620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:50.738028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42252 (local address 127.28.218.193:46511)
0806 22:49:50.738092 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:50.738094 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:50.738193 (+ 99us) server_negotiation.cc:407] Connection header received
0806 22:49:50.738283 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:50.738284 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:50.738294 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:50.738375 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:50.738979 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:50.739500 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:50.740202 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:50.740394 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:50.740470 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:50.742471 (+ 2001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:50.742472 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:50.742472 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:49:50.742479 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:50.743774 (+ 1295us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:50.744142 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:50.744143 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:50.744143 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:50.744202 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:50.744400 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:50.744401 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:50.744401 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:50.744501 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:50.744600 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:50.744739 (+ 139us) server_negotiation.cc:299] Negotiation successful
0806 22:49:50.744814 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
W20260806 22:49:50.745359 30241 tablet_service.cc:2908] Rejecting scan request for tablet e0fc2ccb0381435b9c4e831157639f90: Uninitialized: clean time has not yet been initialized
I20260806 22:49:50.746153 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 30197
I20260806 22:49:50.752528 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 30333
I20260806 22:49:50.757571 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 30472
I20260806 22:49:50.762528 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 30100
2026-08-06T22:49:50Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2037 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056586788339-29547-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:49:50 dist-test-slave-1xrh krb5kdc[30645](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:49:50 dist-test-slave-1xrh krb5kdc[30645](info): set up 2 sockets
Aug 06 22:49:50 dist-test-slave-1xrh krb5kdc[30645](info): commencing operation
krb5kdc: starting...
W20260806 22:49:51.867210 29547 mini_kdc.cc:121] Time spent starting KDC: real 1.083s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056586788339-29547-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056586788339-29547-0/krb5kdc/test-user.keytab.
Aug 06 22:49:51 dist-test-slave-1xrh krb5kdc[30645](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056591, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:49:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:49:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.218.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.218.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.218.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab.
Entry for principal kudu/127.28.218.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab.
I20260806 22:49:51.987031 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056586788339-29547-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056586788339-29547-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056586788339-29547-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.218.254:43095
--webserver_interface=127.28.218.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab
--principal=kudu/127.28.218.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.218.212:44811
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.218.254:43095
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:49:52.076853 30659 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:49:52.077033 30659 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:49:52.077054 30659 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:49:52.078154 30659 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:49:52.078207 30659 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 22:49:52.078220 30659 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:49:52.078233 30659 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:49:52.078281 30659 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:49:52.078298 30659 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:49:52.079806 30659 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:44811
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056586788339-29547-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056586788339-29547-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.218.254:43095
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab
--principal=kudu/127.28.218.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.218.254:43095
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.218.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30659
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056586788339-29547-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:49:52.080035 30659 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:49:52.080261 30659 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:49:52.085378 30665 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:52.085402 30667 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:52.085388 30664 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:49:52.086107 30659 server_base.cc:1034] running on GCE node
I20260806 22:49:52.086365 30659 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:49:52.086656 30659 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:49:52.087808 30659 hybrid_clock.cc:648] HybridClock initialized: now 1786056592087789 us; error 36 us; skew 500 ppm
Aug 06 22:49:52 dist-test-slave-1xrh krb5kdc[30645](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056592, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:49:52.090924 30659 init.cc:377] Logged in from keytab as kudu/127.28.218.254@KRBTEST.COM (short username kudu)
I20260806 22:49:52.091461 30659 webserver.cc:460] Webserver started at http://127.28.218.254:40975/ using document root <none> and password file <none>
I20260806 22:49:52.091730 30659 fs_manager.cc:359] Metadata directory not provided
I20260806 22:49:52.091799 30659 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:49:52.092002 30659 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:49:52.093323 30659 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056586788339-29547-0/minicluster-data/master-0/data/instance:
uuid: "5e2fa0b04c9a4d41a3a5532eb98e4777"
format_stamp: "Formatted at 2026-08-06 22:49:52 on dist-test-slave-1xrh"
server_key: "536ac1cfccff2aa9874341f97183abee"
server_key_iv: "e0b0a91c4f772c222630e6c4bdd3228e"
server_key_version: "encryptionkey@0"
I20260806 22:49:52.093701 30659 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056586788339-29547-0/minicluster-data/master-0/wal/instance:
uuid: "5e2fa0b04c9a4d41a3a5532eb98e4777"
format_stamp: "Formatted at 2026-08-06 22:49:52 on dist-test-slave-1xrh"
server_key: "536ac1cfccff2aa9874341f97183abee"
server_key_iv: "e0b0a91c4f772c222630e6c4bdd3228e"
server_key_version: "encryptionkey@0"
I20260806 22:49:52.095038 30659 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:49:52.095830 30676 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:52.096015 30659 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:49:52.096113 30659 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056586788339-29547-0/minicluster-data/master-0/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056586788339-29547-0/minicluster-data/master-0/wal
uuid: "5e2fa0b04c9a4d41a3a5532eb98e4777"
format_stamp: "Formatted at 2026-08-06 22:49:52 on dist-test-slave-1xrh"
server_key: "536ac1cfccff2aa9874341f97183abee"
server_key_iv: "e0b0a91c4f772c222630e6c4bdd3228e"
server_key_version: "encryptionkey@0"
I20260806 22:49:52.096212 30659 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056586788339-29547-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056586788339-29547-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056586788339-29547-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:49:52.108596 30659 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:49:52.110134 30659 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:49:52.110280 30659 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:49:52.115290 30659 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.254:43095
I20260806 22:49:52.115330 30741 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.254:43095 every 8 connection(s)
I20260806 22:49:52.115645 30659 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
I20260806 22:49:52.116253 30742 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:49:52.118340 30742 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e2fa0b04c9a4d41a3a5532eb98e4777: Bootstrap starting.
I20260806 22:49:52.118881 30742 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5e2fa0b04c9a4d41a3a5532eb98e4777: Neither blocks nor log segments found. Creating new log.
I20260806 22:49:52.119161 30742 log.cc:824] T 00000000000000000000000000000000 P 5e2fa0b04c9a4d41a3a5532eb98e4777: Log is configured to *not* fsync() on all Append() calls
I20260806 22:49:52.119776 30742 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e2fa0b04c9a4d41a3a5532eb98e4777: No bootstrap required, opened a new log
I20260806 22:49:52.121766 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 30659
I20260806 22:49:52.121865 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056586788339-29547-0/minicluster-data/master-0/wal/instance
I20260806 22:49:52.122006 29547 external_mini_cluster.cc:1428] Setting key 7940ebe5e6d50083ad696bd35ba981c4
I20260806 22:49:52.121966 30742 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5e2fa0b04c9a4d41a3a5532eb98e4777 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e2fa0b04c9a4d41a3a5532eb98e4777" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 43095 } }
I20260806 22:49:52.122117 30742 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5e2fa0b04c9a4d41a3a5532eb98e4777 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:49:52.122184 30742 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5e2fa0b04c9a4d41a3a5532eb98e4777 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e2fa0b04c9a4d41a3a5532eb98e4777, State: Initialized, Role: FOLLOWER
I20260806 22:49:52.122339 30742 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5e2fa0b04c9a4d41a3a5532eb98e4777 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e2fa0b04c9a4d41a3a5532eb98e4777" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 43095 } }
I20260806 22:49:52.122428 30742 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5e2fa0b04c9a4d41a3a5532eb98e4777 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:49:52.122468 30742 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5e2fa0b04c9a4d41a3a5532eb98e4777 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:49:52.122566 30742 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5e2fa0b04c9a4d41a3a5532eb98e4777 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:49:52.123237 30742 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5e2fa0b04c9a4d41a3a5532eb98e4777 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e2fa0b04c9a4d41a3a5532eb98e4777" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 43095 } }
I20260806 22:49:52.123350 30742 leader_election.cc:302] T 00000000000000000000000000000000 P 5e2fa0b04c9a4d41a3a5532eb98e4777 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5e2fa0b04c9a4d41a3a5532eb98e4777; no voters:
I20260806 22:49:52.123512 30742 leader_election.cc:288] T 00000000000000000000000000000000 P 5e2fa0b04c9a4d41a3a5532eb98e4777 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:49:52.123539 30749 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5e2fa0b04c9a4d41a3a5532eb98e4777 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:49:52.123739 30749 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5e2fa0b04c9a4d41a3a5532eb98e4777 [term 1 LEADER]: Becoming Leader. State: Replica: 5e2fa0b04c9a4d41a3a5532eb98e4777, State: Running, Role: LEADER
I20260806 22:49:52.123785 30742 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5e2fa0b04c9a4d41a3a5532eb98e4777 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:49:52.123870 30749 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5e2fa0b04c9a4d41a3a5532eb98e4777 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e2fa0b04c9a4d41a3a5532eb98e4777" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 43095 } }
I20260806 22:49:52.124214 30749 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e2fa0b04c9a4d41a3a5532eb98e4777 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5e2fa0b04c9a4d41a3a5532eb98e4777. Latest consensus state: current_term: 1 leader_uuid: "5e2fa0b04c9a4d41a3a5532eb98e4777" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e2fa0b04c9a4d41a3a5532eb98e4777" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 43095 } } }
I20260806 22:49:52.124287 30749 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e2fa0b04c9a4d41a3a5532eb98e4777 [sys.catalog]: This master's current role is: LEADER
I20260806 22:49:52.125048 30755 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:49:52.125568 30750 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e2fa0b04c9a4d41a3a5532eb98e4777 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5e2fa0b04c9a4d41a3a5532eb98e4777" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e2fa0b04c9a4d41a3a5532eb98e4777" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 43095 } } }
I20260806 22:49:52.125648 30750 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e2fa0b04c9a4d41a3a5532eb98e4777 [sys.catalog]: This master's current role is: LEADER
I20260806 22:49:52.126434 30755 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:49:52.128340 30755 catalog_manager.cc:1347] Generated new cluster ID: f28bea9335e5414d91d79a3076609492
I20260806 22:49:52.128392 30755 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 22:49:52 dist-test-slave-1xrh krb5kdc[30645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056591, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.218.254@KRBTEST.COM
I20260806 22:49:52.133033 30747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:52.122393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45626 (local address 127.28.218.254:43095)
0806 22:49:52.122566 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:52.122575 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:52.122695 (+ 120us) server_negotiation.cc:407] Connection header received
0806 22:49:52.122823 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:52.122828 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:52.122866 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:52.122975 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:52.123886 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:52.125248 (+ 1362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:52.126127 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:52.126551 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:52.126680 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:52.129287 (+ 2607us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:52.129291 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:52.129291 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:49:52.129302 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:52.131350 (+ 2048us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:52.131715 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:52.131718 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:52.131718 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:52.131834 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:52.132122 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:52.132123 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:52.132123 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:52.132360 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:52.132524 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:52.132619 (+ 95us) server_negotiation.cc:299] Negotiation successful
0806 22:49:52.132761 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":92,"threads_started":1}
I20260806 22:49:52.138520 30755 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:49:52.139219 30755 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:49:52.155138 30755 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5e2fa0b04c9a4d41a3a5532eb98e4777: Generated new TSK 0
I20260806 22:49:52.155316 30755 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.28.218.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.218.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.218.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.193.keytab.
Entry for principal kudu/127.28.218.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.193.keytab.
I20260806 22:49:52.187489 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056586788339-29547-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056586788339-29547-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056586788339-29547-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056586788339-29547-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.218.193:0
--local_ip_for_outbound_sockets=127.28.218.193
--webserver_interface=127.28.218.193
--webserver_port=0
--tserver_master_addrs=127.28.218.254:43095
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.193.keytab
--principal=kudu/127.28.218.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.218.212:44811
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:49:52.267928 30770 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:49:52.268115 30770 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:49:52.268138 30770 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:49:52.269203 30770 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:49:52.269235 30770 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:49:52.269256 30770 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 22:49:52.269273 30770 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.218.193
I20260806 22:49:52.270576 30770 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:44811
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056586788339-29547-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056586788339-29547-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.193.keytab
--principal=kudu/127.28.218.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.218.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056586788339-29547-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.218.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.28.218.254:43095
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30770
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.218.193
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056586788339-29547-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:49:52.270735 30770 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:49:52.270933 30770 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:49:52.275801 30775 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:52.275817 30778 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:49:52.275866 30770 server_base.cc:1034] running on GCE node
W20260806 22:49:52.275846 30776 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:49:52.276165 30770 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:49:52.276340 30770 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:49:52.277468 30770 hybrid_clock.cc:648] HybridClock initialized: now 1786056592277459 us; error 26 us; skew 500 ppm
Aug 06 22:49:52 dist-test-slave-1xrh krb5kdc[30645](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056592, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:49:52.279989 30770 init.cc:377] Logged in from keytab as kudu/127.28.218.193@KRBTEST.COM (short username kudu)
I20260806 22:49:52.280390 30770 webserver.cc:460] Webserver started at http://127.28.218.193:38851/ using document root <none> and password file <none>
I20260806 22:49:52.280608 30770 fs_manager.cc:359] Metadata directory not provided
I20260806 22:49:52.280678 30770 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:49:52.280826 30770 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:49:52.281831 30770 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056586788339-29547-0/minicluster-data/ts-0/data/instance:
uuid: "f08aecdfef264c8b81b70ff01cbed9cd"
format_stamp: "Formatted at 2026-08-06 22:49:52 on dist-test-slave-1xrh"
server_key: "66260e99cc0c87a4275d44307fd5fefc"
server_key_iv: "d77f2cbe14e93795d3a2c623864ba46a"
server_key_version: "encryptionkey@0"
I20260806 22:49:52.282191 30770 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056586788339-29547-0/minicluster-data/ts-0/wal/instance:
uuid: "f08aecdfef264c8b81b70ff01cbed9cd"
format_stamp: "Formatted at 2026-08-06 22:49:52 on dist-test-slave-1xrh"
server_key: "66260e99cc0c87a4275d44307fd5fefc"
server_key_iv: "d77f2cbe14e93795d3a2c623864ba46a"
server_key_version: "encryptionkey@0"
I20260806 22:49:52.283548 30770 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:49:52.284283 30785 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:52.284461 30770 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:49:52.284560 30770 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056586788339-29547-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056586788339-29547-0/minicluster-data/ts-0/wal
uuid: "f08aecdfef264c8b81b70ff01cbed9cd"
format_stamp: "Formatted at 2026-08-06 22:49:52 on dist-test-slave-1xrh"
server_key: "66260e99cc0c87a4275d44307fd5fefc"
server_key_iv: "d77f2cbe14e93795d3a2c623864ba46a"
server_key_version: "encryptionkey@0"
I20260806 22:49:52.284644 30770 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056586788339-29547-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056586788339-29547-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056586788339-29547-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:49:52.321095 30770 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:49:52.323495 30770 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:49:52.323658 30770 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:49:52.324158 30770 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:49:52.324219 30770 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:52.324255 30770 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:49:52.324323 30770 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:52.331842 30770 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.193:45763
I20260806 22:49:52.331921 30904 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.193:45763 every 8 connection(s)
I20260806 22:49:52.332185 30770 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056586788339-29547-0/minicluster-data/ts-0/data/info.pb
Aug 06 22:49:52 dist-test-slave-1xrh krb5kdc[30645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056592, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.193@KRBTEST.COM for kudu/127.28.218.254@KRBTEST.COM
I20260806 22:49:52.332419 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 30770
I20260806 22:49:52.332525 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056586788339-29547-0/minicluster-data/ts-0/wal/instance
I20260806 22:49:52.332697 29547 external_mini_cluster.cc:1428] Setting key 4c0c24b3e626ad8e0d776e1a55ffd4d6
I20260806 22:49:52.335901 30747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:52.324867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.193:44149 (local address 127.28.218.254:43095)
0806 22:49:52.324931 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:52.324933 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:52.325161 (+ 228us) server_negotiation.cc:407] Connection header received
0806 22:49:52.325789 (+ 628us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:52.325791 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:52.325809 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:52.325875 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:52.326768 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:52.327638 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:52.329252 (+ 1614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:52.329481 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:52.329541 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:52.332812 (+ 3271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:52.332813 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:52.332813 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:49:52.332820 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:52.334427 (+ 1607us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:52.334878 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:52.334879 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:52.334879 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:52.334943 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:52.335234 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:52.335235 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:52.335236 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:52.335355 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:52.335433 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:52.335713 (+ 280us) server_negotiation.cc:299] Negotiation successful
0806 22:49:52.335802 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 22:49:52.336120 30794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:52.324872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:43095 (local address 127.28.218.193:44149)
0806 22:49:52.325047 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:52.325080 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:52.325691 (+ 611us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:52.325980 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:52.325984 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:52.326373 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:49:52.326667 (+ 294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:52.326674 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:52.327709 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:52.327710 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:52.329154 (+ 1444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:52.329157 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:52.329573 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:52.329574 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:52.329733 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:52.330831 (+ 1098us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:49:52.330842 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:49:52.332687 (+ 1845us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:49:52.334546 (+ 1859us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:52.334553 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:52.334555 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:52.334775 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:52.335010 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:52.335011 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:52.335012 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:52.335168 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:52.335423 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:49:52.335429 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:49:52.335564 (+ 135us) client_negotiation.cc:769] Sending connection context
0806 22:49:52.335686 (+ 122us) client_negotiation.cc:240] Negotiation successful
0806 22:49:52.335888 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":102,"threads_started":1}
I20260806 22:49:52.336566 30905 heartbeater.cc:347] Connected to a master server at 127.28.218.254:43095
I20260806 22:49:52.336699 30905 heartbeater.cc:464] Registering TS with master...
I20260806 22:49:52.336923 30905 heartbeater.cc:511] Master 127.28.218.254:43095 requested a full tablet report, sending...
I20260806 22:49:52.337517 30693 ts_manager.cc:194] Registered new tserver with Master: f08aecdfef264c8b81b70ff01cbed9cd (127.28.218.193:45763)
I20260806 22:49:52.338475 30693 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.218.193@KRBTEST.COM'} at 127.28.218.193:44149
WARNING: no policy specified for kudu/127.28.218.194@KRBTEST.COM; defaulting to no policy
I20260806 22:49:52.343544 30747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:52.337665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.193:53809 (local address 127.28.218.254:43095)
0806 22:49:52.337701 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:52.337703 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:52.337871 (+ 168us) server_negotiation.cc:407] Connection header received
0806 22:49:52.337940 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:52.337941 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:52.337951 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:52.338005 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:52.338305 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:52.338774 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:52.339753 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:52.339963 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:52.340048 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:52.340936 (+ 888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:52.340938 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:52.340938 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:49:52.340944 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:52.342322 (+ 1378us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:52.342724 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:52.342726 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:52.342726 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:52.342779 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:52.343003 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:52.343004 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:52.343004 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:52.343111 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:52.343211 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:52.343384 (+ 173us) server_negotiation.cc:299] Negotiation successful
0806 22:49:52.343457 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260806 22:49:52.343560 30912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:52.337619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:43095 (local address 127.28.218.193:53809)
0806 22:49:52.337792 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:52.337801 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:52.337883 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:52.338006 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:52.338007 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:52.338185 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:49:52.338229 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:52.338230 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:52.338875 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:52.338876 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:52.339666 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:52.339668 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:52.340049 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:52.340050 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:52.340117 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:52.340548 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:49:52.340553 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:49:52.340832 (+ 279us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:49:52.342453 (+ 1621us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:52.342454 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:52.342455 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:52.342623 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:52.342875 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:52.342876 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:52.342877 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:52.342940 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:52.343203 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:49:52.343204 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:49:52.343247 (+ 43us) client_negotiation.cc:769] Sending connection context
0806 22:49:52.343356 (+ 109us) client_negotiation.cc:240] Negotiation successful
0806 22:49:52.343457 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":128,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.218.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.218.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.194.keytab.
Entry for principal kudu/127.28.218.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.194.keytab.
I20260806 22:49:52.362546 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056586788339-29547-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056586788339-29547-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056586788339-29547-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056586788339-29547-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.218.194:0
--local_ip_for_outbound_sockets=127.28.218.194
--webserver_interface=127.28.218.194
--webserver_port=0
--tserver_master_addrs=127.28.218.254:43095
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.194.keytab
--principal=kudu/127.28.218.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.218.212:44811
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:49:52.449649 30915 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:49:52.449859 30915 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:49:52.449905 30915 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:49:52.450961 30915 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:49:52.450994 30915 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:49:52.451013 30915 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 22:49:52.451030 30915 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.218.194
I20260806 22:49:52.452373 30915 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:44811
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056586788339-29547-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056586788339-29547-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.194.keytab
--principal=kudu/127.28.218.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.218.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056586788339-29547-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.218.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.28.218.254:43095
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30915
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.218.194
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056586788339-29547-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:49:52.452541 30915 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:49:52.452700 30915 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:49:52.457616 30921 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:52.457653 30920 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:49:52.457777 30915 server_base.cc:1034] running on GCE node
W20260806 22:49:52.457661 30923 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:49:52.458081 30915 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:49:52.458258 30915 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:49:52.459388 30915 hybrid_clock.cc:648] HybridClock initialized: now 1786056592459374 us; error 30 us; skew 500 ppm
Aug 06 22:49:52 dist-test-slave-1xrh krb5kdc[30645](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056592, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:49:52.461781 30915 init.cc:377] Logged in from keytab as kudu/127.28.218.194@KRBTEST.COM (short username kudu)
I20260806 22:49:52.462263 30915 webserver.cc:460] Webserver started at http://127.28.218.194:45535/ using document root <none> and password file <none>
I20260806 22:49:52.462507 30915 fs_manager.cc:359] Metadata directory not provided
I20260806 22:49:52.462575 30915 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:49:52.462728 30915 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:49:52.463752 30915 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056586788339-29547-0/minicluster-data/ts-1/data/instance:
uuid: "36075fa9435445349d1c8429845ed592"
format_stamp: "Formatted at 2026-08-06 22:49:52 on dist-test-slave-1xrh"
server_key: "840178d70b73bc5f2e1b400f6c0e5405"
server_key_iv: "7497bd296f89268693ab3a553876b49b"
server_key_version: "encryptionkey@0"
I20260806 22:49:52.464123 30915 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056586788339-29547-0/minicluster-data/ts-1/wal/instance:
uuid: "36075fa9435445349d1c8429845ed592"
format_stamp: "Formatted at 2026-08-06 22:49:52 on dist-test-slave-1xrh"
server_key: "840178d70b73bc5f2e1b400f6c0e5405"
server_key_iv: "7497bd296f89268693ab3a553876b49b"
server_key_version: "encryptionkey@0"
I20260806 22:49:52.465387 30915 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:49:52.466075 30930 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:52.466239 30915 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:49:52.466336 30915 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056586788339-29547-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056586788339-29547-0/minicluster-data/ts-1/wal
uuid: "36075fa9435445349d1c8429845ed592"
format_stamp: "Formatted at 2026-08-06 22:49:52 on dist-test-slave-1xrh"
server_key: "840178d70b73bc5f2e1b400f6c0e5405"
server_key_iv: "7497bd296f89268693ab3a553876b49b"
server_key_version: "encryptionkey@0"
I20260806 22:49:52.466419 30915 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056586788339-29547-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056586788339-29547-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056586788339-29547-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:49:52.481789 30915 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:49:52.483577 30915 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:49:52.483722 30915 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:49:52.484256 30915 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:49:52.484313 30915 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:52.484354 30915 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:49:52.484402 30915 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 22:49:52 dist-test-slave-1xrh krb5kdc[30645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056592, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.194@KRBTEST.COM for kudu/127.28.218.254@KRBTEST.COM
I20260806 22:49:52.491490 30915 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.194:34297
I20260806 22:49:52.491550 31049 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.194:34297 every 8 connection(s)
I20260806 22:49:52.491851 30915 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056586788339-29547-0/minicluster-data/ts-1/data/info.pb
I20260806 22:49:52.494321 30747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:52.484641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.194:42771 (local address 127.28.218.254:43095)
0806 22:49:52.484696 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:52.484698 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:52.484945 (+ 247us) server_negotiation.cc:407] Connection header received
0806 22:49:52.485614 (+ 669us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:52.485615 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:52.485626 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:52.485667 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:52.486521 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:52.487385 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:52.488943 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:52.489286 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:52.489371 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:52.492007 (+ 2636us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:52.492008 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:52.492008 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:49:52.492014 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:52.493259 (+ 1245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:52.493595 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:52.493596 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:52.493596 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:52.493644 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:52.493835 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:52.493836 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:52.493836 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:52.493929 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:52.493976 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:52.494205 (+ 229us) server_negotiation.cc:299] Negotiation successful
0806 22:49:52.494260 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260806 22:49:52.494470 30939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:52.484623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:43095 (local address 127.28.218.194:42771)
0806 22:49:52.484782 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:52.484833 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:52.485509 (+ 676us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:52.485740 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:52.485745 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:52.486143 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:49:52.486435 (+ 292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:52.486442 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:52.487458 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:52.487459 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:52.488844 (+ 1385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:52.488848 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:52.489402 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:52.489403 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:52.489544 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:52.490070 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:49:52.490078 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:49:52.491917 (+ 1839us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:49:52.493324 (+ 1407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:52.493329 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:52.493331 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:52.493532 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:52.493698 (+ 166us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:52.493699 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:52.493699 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:52.493784 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:52.493994 (+ 210us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:49:52.493997 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:49:52.494093 (+ 96us) client_negotiation.cc:769] Sending connection context
0806 22:49:52.494189 (+ 96us) client_negotiation.cc:240] Negotiation successful
0806 22:49:52.494296 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":77,"threads_started":1}
I20260806 22:49:52.494947 31050 heartbeater.cc:347] Connected to a master server at 127.28.218.254:43095
I20260806 22:49:52.495054 31050 heartbeater.cc:464] Registering TS with master...
I20260806 22:49:52.495244 31050 heartbeater.cc:511] Master 127.28.218.254:43095 requested a full tablet report, sending...
I20260806 22:49:52.495673 30694 ts_manager.cc:194] Registered new tserver with Master: 36075fa9435445349d1c8429845ed592 (127.28.218.194:34297)
I20260806 22:49:52.496207 30694 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.218.194@KRBTEST.COM'} at 127.28.218.194:42771
I20260806 22:49:52.497756 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 30915
I20260806 22:49:52.497850 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056586788339-29547-0/minicluster-data/ts-1/wal/instance
I20260806 22:49:52.498134 29547 external_mini_cluster.cc:1428] Setting key ae2b52fd2159967504316a2546247e2f
I20260806 22:49:52.502215 31057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:52.495921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:43095 (local address 127.28.218.194:40527)
0806 22:49:52.496058 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:52.496069 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:52.496136 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:52.496240 (+ 104us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:52.496241 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:52.496440 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:49:52.496508 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:52.496510 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:52.497244 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:52.497245 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:52.497891 (+ 646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:52.497894 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:52.498430 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:52.498431 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:52.498508 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:52.498940 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:49:52.498945 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:49:52.499276 (+ 331us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:49:52.500951 (+ 1675us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:52.500952 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:52.500953 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:52.501183 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:52.501368 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:52.501369 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:52.501369 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:52.501437 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:52.501982 (+ 545us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:49:52.501983 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:49:52.502021 (+ 38us) client_negotiation.cc:769] Sending connection context
0806 22:49:52.502079 (+ 58us) client_negotiation.cc:240] Negotiation successful
0806 22:49:52.502149 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":89,"threads_started":1}
I20260806 22:49:52.502321 30747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:52.496010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.194:40527 (local address 127.28.218.254:43095)
0806 22:49:52.496057 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:52.496059 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:52.496129 (+ 70us) server_negotiation.cc:407] Connection header received
0806 22:49:52.496186 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:52.496187 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:52.496194 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:52.496234 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:52.496587 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:52.497166 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:52.498046 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:52.498322 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:52.498413 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:52.499351 (+ 938us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:52.499353 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:52.499353 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:49:52.499360 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:52.500872 (+ 1512us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:52.501258 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:52.501259 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:52.501259 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:52.501305 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:52.501665 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:52.501666 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:52.501667 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:52.501901 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:52.501995 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:52.502104 (+ 109us) server_negotiation.cc:299] Negotiation successful
0806 22:49:52.502188 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
WARNING: no policy specified for kudu/127.28.218.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.218.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.218.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.195.keytab.
Entry for principal kudu/127.28.218.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.195.keytab.
I20260806 22:49:52.525923 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056586788339-29547-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056586788339-29547-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056586788339-29547-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056586788339-29547-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.218.195:0
--local_ip_for_outbound_sockets=127.28.218.195
--webserver_interface=127.28.218.195
--webserver_port=0
--tserver_master_addrs=127.28.218.254:43095
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.195.keytab
--principal=kudu/127.28.218.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.218.212:44811
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:49:52.606599 31060 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:49:52.606797 31060 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:49:52.606822 31060 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:49:52.608044 31060 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:49:52.608086 31060 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:49:52.608107 31060 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 22:49:52.608170 31060 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.218.195
I20260806 22:49:52.609570 31060 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:44811
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056586788339-29547-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056586788339-29547-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.195.keytab
--principal=kudu/127.28.218.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.218.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056586788339-29547-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.218.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.28.218.254:43095
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31060
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.218.195
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056586788339-29547-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:49:52.609805 31060 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:49:52.610030 31060 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:49:52.614795 31066 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:52.614805 31065 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:49:52.614837 31060 server_base.cc:1034] running on GCE node
W20260806 22:49:52.614846 31068 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:49:52.615237 31060 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:49:52.615473 31060 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:49:52.616649 31060 hybrid_clock.cc:648] HybridClock initialized: now 1786056592616631 us; error 28 us; skew 500 ppm
Aug 06 22:49:52 dist-test-slave-1xrh krb5kdc[30645](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056592, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:49:52.618944 31060 init.cc:377] Logged in from keytab as kudu/127.28.218.195@KRBTEST.COM (short username kudu)
I20260806 22:49:52.619428 31060 webserver.cc:460] Webserver started at http://127.28.218.195:39899/ using document root <none> and password file <none>
I20260806 22:49:52.619697 31060 fs_manager.cc:359] Metadata directory not provided
I20260806 22:49:52.619766 31060 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:49:52.619915 31060 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:49:52.621124 31060 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056586788339-29547-0/minicluster-data/ts-2/data/instance:
uuid: "998381aa0f734eec9531d0ce425fbe96"
format_stamp: "Formatted at 2026-08-06 22:49:52 on dist-test-slave-1xrh"
server_key: "598a6eb3bb1ecfd969a3de795829229b"
server_key_iv: "a4573238f592ae8f24d19cb0e4840476"
server_key_version: "encryptionkey@0"
I20260806 22:49:52.621523 31060 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056586788339-29547-0/minicluster-data/ts-2/wal/instance:
uuid: "998381aa0f734eec9531d0ce425fbe96"
format_stamp: "Formatted at 2026-08-06 22:49:52 on dist-test-slave-1xrh"
server_key: "598a6eb3bb1ecfd969a3de795829229b"
server_key_iv: "a4573238f592ae8f24d19cb0e4840476"
server_key_version: "encryptionkey@0"
I20260806 22:49:52.622802 31060 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:49:52.623651 31075 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:52.623829 31060 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 22:49:52.623925 31060 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056586788339-29547-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056586788339-29547-0/minicluster-data/ts-2/wal
uuid: "998381aa0f734eec9531d0ce425fbe96"
format_stamp: "Formatted at 2026-08-06 22:49:52 on dist-test-slave-1xrh"
server_key: "598a6eb3bb1ecfd969a3de795829229b"
server_key_iv: "a4573238f592ae8f24d19cb0e4840476"
server_key_version: "encryptionkey@0"
I20260806 22:49:52.624037 31060 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056586788339-29547-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056586788339-29547-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056586788339-29547-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:49:52.634994 31060 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:49:52.636773 31060 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:49:52.636920 31060 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:49:52.637408 31060 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:49:52.637460 31060 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:52.637533 31060 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:49:52.637563 31060 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 22:49:52 dist-test-slave-1xrh krb5kdc[30645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056592, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.195@KRBTEST.COM for kudu/127.28.218.254@KRBTEST.COM
I20260806 22:49:52.644964 31060 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.195:45779
I20260806 22:49:52.645051 31195 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.195:45779 every 8 connection(s)
I20260806 22:49:52.645339 31060 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056586788339-29547-0/minicluster-data/ts-2/data/info.pb
I20260806 22:49:52.646263 30747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:52.637736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.195:60731 (local address 127.28.218.254:43095)
0806 22:49:52.637788 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:52.637791 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:52.638024 (+ 233us) server_negotiation.cc:407] Connection header received
0806 22:49:52.638449 (+ 425us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:52.638451 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:52.638466 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:52.638571 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:52.639210 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:52.639755 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:52.640748 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:52.640954 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:52.641027 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:52.643554 (+ 2527us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:52.643555 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:52.643556 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:49:52.643564 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:52.644936 (+ 1372us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:52.645373 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:52.645374 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:52.645375 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:52.645433 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:52.645736 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:52.645737 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:52.645737 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:52.645865 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:52.645915 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:52.646141 (+ 226us) server_negotiation.cc:299] Negotiation successful
0806 22:49:52.646203 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 22:49:52.646422 31085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:52.637732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:43095 (local address 127.28.218.195:60731)
0806 22:49:52.637877 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:52.637912 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:52.638347 (+ 435us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:52.638613 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:52.638616 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:52.638831 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:49:52.639056 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:52.639062 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:52.639847 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:52.639848 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:52.640659 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:52.640662 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:52.641046 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:52.641047 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:52.641194 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:52.641692 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:49:52.641704 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:49:52.643459 (+ 1755us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:49:52.645058 (+ 1599us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:52.645062 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:52.645064 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:52.645294 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:52.645505 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:52.645506 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:52.645506 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:52.645644 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:52.645919 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:49:52.645921 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:49:52.646023 (+ 102us) client_negotiation.cc:769] Sending connection context
0806 22:49:52.646111 (+ 88us) client_negotiation.cc:240] Negotiation successful
0806 22:49:52.646218 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":87,"threads_started":1}
I20260806 22:49:52.646903 31196 heartbeater.cc:347] Connected to a master server at 127.28.218.254:43095
I20260806 22:49:52.646991 31196 heartbeater.cc:464] Registering TS with master...
I20260806 22:49:52.647207 31196 heartbeater.cc:511] Master 127.28.218.254:43095 requested a full tablet report, sending...
I20260806 22:49:52.647575 30693 ts_manager.cc:194] Registered new tserver with Master: 998381aa0f734eec9531d0ce425fbe96 (127.28.218.195:45779)
I20260806 22:49:52.648073 30693 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.218.195@KRBTEST.COM'} at 127.28.218.195:60731
I20260806 22:49:52.650550 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 31060
I20260806 22:49:52.650635 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056586788339-29547-0/minicluster-data/ts-2/wal/instance
I20260806 22:49:52.650790 29547 external_mini_cluster.cc:1428] Setting key 73a044999134e5f34389f453720308b1
I20260806 22:49:52.651526 29547 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:49:52.653316 30747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:52.647998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.195:48097 (local address 127.28.218.254:43095)
0806 22:49:52.648038 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:52.648040 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:52.648170 (+ 130us) server_negotiation.cc:407] Connection header received
0806 22:49:52.648230 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:52.648231 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:52.648241 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:52.648308 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:52.648612 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:52.649050 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:52.649719 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:52.649909 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:52.649980 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:52.650842 (+ 862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:52.650844 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:52.650844 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:49:52.650851 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:52.652223 (+ 1372us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:52.652605 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:52.652605 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:52.652606 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:52.652654 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:52.652844 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:52.652844 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:52.652845 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:52.652997 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:52.653061 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:52.653210 (+ 149us) server_negotiation.cc:299] Negotiation successful
0806 22:49:52.653264 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 22:49:52.653436 31202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:52.647913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:43095 (local address 127.28.218.195:48097)
0806 22:49:52.648093 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:52.648104 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:52.648166 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:52.648320 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:52.648320 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:52.648472 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:49:52.648526 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:52.648527 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:52.649152 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:52.649153 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:52.649627 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:52.649628 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:52.649980 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:52.649981 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:52.650050 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:52.650400 (+ 350us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:49:52.650405 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:49:52.650748 (+ 343us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:49:52.652303 (+ 1555us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:52.652303 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:52.652304 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:52.652530 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:52.652717 (+ 187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:52.652717 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:52.652718 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:52.652785 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:52.653071 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:49:52.653072 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:49:52.653115 (+ 43us) client_negotiation.cc:769] Sending connection context
0806 22:49:52.653186 (+ 71us) client_negotiation.cc:240] Negotiation successful
0806 22:49:52.653289 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":89,"threads_started":1}
Aug 06 22:49:52 dist-test-slave-1xrh krb5kdc[30645](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056592, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 22:49:52 dist-test-slave-1xrh krb5kdc[30645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056592, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.28.218.254@KRBTEST.COM
I20260806 22:49:52.674619 30747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:52.668028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45642 (local address 127.28.218.254:43095)
0806 22:49:52.668085 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:52.668087 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:52.668103 (+ 16us) server_negotiation.cc:407] Connection header received
0806 22:49:52.668147 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:52.668148 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:52.668155 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:52.668200 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:52.668765 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:52.669297 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:52.669974 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:52.670192 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:52.670242 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:52.672449 (+ 2207us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:52.672450 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:52.672450 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:49:52.672455 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:52.673518 (+ 1063us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:52.673857 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:52.673858 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:52.673858 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:52.673913 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:52.674122 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:52.674123 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:52.674123 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:52.674221 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:52.674316 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:52.674430 (+ 114us) server_negotiation.cc:299] Negotiation successful
0806 22:49:52.674504 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 22:49:52.676047 30693 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45642:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:49:52.677465 30693 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:49:52.683830 31213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:52.679617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.193:45763 (local address 127.0.0.1:47754)
0806 22:49:52.679927 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:52.679937 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:52.679991 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:52.680666 (+ 675us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:52.680667 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:52.680669 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:49:52.680706 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:52.680708 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:52.681887 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:52.681888 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:52.682764 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:52.682767 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:52.683524 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:52.683525 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:52.683603 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:52.683615 (+ 12us) client_negotiation.cc:769] Sending connection context
0806 22:49:52.683687 (+ 72us) client_negotiation.cc:240] Negotiation successful
0806 22:49:52.683737 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":84,"threads_started":1}
I20260806 22:49:52.683853 31215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:52.679723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47754 (local address 127.28.218.193:45763)
0806 22:49:52.680445 (+ 722us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:52.680450 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:52.680468 (+ 18us) server_negotiation.cc:407] Connection header received
0806 22:49:52.680504 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:52.680506 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:52.680521 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:52.680593 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:49:52.680772 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:52.681826 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:52.682846 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:52.683419 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:52.683524 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:52.683584 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:52.683709 (+ 125us) server_negotiation.cc:299] Negotiation successful
0806 22:49:52.683745 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":687,"thread_start_us":84,"threads_started":1}
I20260806 22:49:52.683984 31214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:52.679602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.195:45779 (local address 127.0.0.1:55384)
0806 22:49:52.679750 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:52.679791 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:52.679860 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:52.680322 (+ 462us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:52.680325 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:52.680327 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:49:52.680391 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:52.680393 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:52.681506 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:52.681508 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:52.682801 (+ 1293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:52.682804 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:52.683739 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:52.683741 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:52.683828 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:52.683832 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:49:52.683865 (+ 33us) client_negotiation.cc:240] Negotiation successful
0806 22:49:52.683904 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"mutex_wait_us":12,"thread_start_us":43,"threads_started":1}
I20260806 22:49:52.684218 31218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:52.679949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52420 (local address 127.28.218.194:34297)
0806 22:49:52.680141 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:52.680144 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:52.680153 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:49:52.680182 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:52.680184 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:52.680195 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:52.680301 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:49:52.680503 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:52.681627 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:52.682900 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:52.683704 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:52.683764 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:52.683833 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:52.684071 (+ 238us) server_negotiation.cc:299] Negotiation successful
0806 22:49:52.684113 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":104,"threads_started":1}
I20260806 22:49:52.684279 31217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:52.679768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55384 (local address 127.28.218.195:45779)
0806 22:49:52.680123 (+ 355us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:52.680127 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:52.680135 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:49:52.680170 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:52.680177 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:52.680188 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:52.680304 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:49:52.680592 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:52.681417 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:52.682900 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:52.683636 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:52.684059 (+ 423us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:52.684115 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:52.684140 (+ 25us) server_negotiation.cc:299] Negotiation successful
0806 22:49:52.684179 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":326,"thread_start_us":89,"threads_started":1}
I20260806 22:49:52.684136 31216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:52.679775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.194:34297 (local address 127.0.0.1:52420)
0806 22:49:52.679956 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:52.679966 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:52.680019 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:52.680375 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:52.680376 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:52.680377 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:49:52.680421 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:52.680422 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:52.681709 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:52.681710 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:52.682802 (+ 1092us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:52.682804 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:52.683867 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:52.683868 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:52.683979 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:52.683982 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:49:52.684042 (+ 60us) client_negotiation.cc:240] Negotiation successful
0806 22:49:52.684077 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"mutex_wait_us":25,"thread_start_us":53,"threads_started":1}
I20260806 22:49:52.684890 31127 tablet_service.cc:1467] Processing CreateTablet for tablet 8f3870fb45db42dcbdfc273e8d735e2c (DEFAULT_TABLE table=test-table [id=af3eaf639ff94e8798763c79de537bb5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:49:52.684927 30837 tablet_service.cc:1467] Processing CreateTablet for tablet 8f3870fb45db42dcbdfc273e8d735e2c (DEFAULT_TABLE table=test-table [id=af3eaf639ff94e8798763c79de537bb5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:49:52.685134 30981 tablet_service.cc:1467] Processing CreateTablet for tablet 8f3870fb45db42dcbdfc273e8d735e2c (DEFAULT_TABLE table=test-table [id=af3eaf639ff94e8798763c79de537bb5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:49:52.685266 30837 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f3870fb45db42dcbdfc273e8d735e2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:49:52.685256 31127 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f3870fb45db42dcbdfc273e8d735e2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:49:52.685432 30981 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f3870fb45db42dcbdfc273e8d735e2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:49:52.688864 31220 tablet_bootstrap.cc:492] T 8f3870fb45db42dcbdfc273e8d735e2c P f08aecdfef264c8b81b70ff01cbed9cd: Bootstrap starting.
I20260806 22:49:52.688920 31219 tablet_bootstrap.cc:492] T 8f3870fb45db42dcbdfc273e8d735e2c P 998381aa0f734eec9531d0ce425fbe96: Bootstrap starting.
I20260806 22:49:52.689476 31220 tablet_bootstrap.cc:654] T 8f3870fb45db42dcbdfc273e8d735e2c P f08aecdfef264c8b81b70ff01cbed9cd: Neither blocks nor log segments found. Creating new log.
I20260806 22:49:52.689759 31219 tablet_bootstrap.cc:654] T 8f3870fb45db42dcbdfc273e8d735e2c P 998381aa0f734eec9531d0ce425fbe96: Neither blocks nor log segments found. Creating new log.
I20260806 22:49:52.689800 31220 log.cc:824] T 8f3870fb45db42dcbdfc273e8d735e2c P f08aecdfef264c8b81b70ff01cbed9cd: Log is configured to *not* fsync() on all Append() calls
I20260806 22:49:52.690107 31221 tablet_bootstrap.cc:492] T 8f3870fb45db42dcbdfc273e8d735e2c P 36075fa9435445349d1c8429845ed592: Bootstrap starting.
I20260806 22:49:52.690101 31219 log.cc:824] T 8f3870fb45db42dcbdfc273e8d735e2c P 998381aa0f734eec9531d0ce425fbe96: Log is configured to *not* fsync() on all Append() calls
I20260806 22:49:52.690507 31220 tablet_bootstrap.cc:492] T 8f3870fb45db42dcbdfc273e8d735e2c P f08aecdfef264c8b81b70ff01cbed9cd: No bootstrap required, opened a new log
I20260806 22:49:52.690588 31220 ts_tablet_manager.cc:1397] T 8f3870fb45db42dcbdfc273e8d735e2c P f08aecdfef264c8b81b70ff01cbed9cd: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:49:52.690801 31221 tablet_bootstrap.cc:654] T 8f3870fb45db42dcbdfc273e8d735e2c P 36075fa9435445349d1c8429845ed592: Neither blocks nor log segments found. Creating new log.
I20260806 22:49:52.690999 31219 tablet_bootstrap.cc:492] T 8f3870fb45db42dcbdfc273e8d735e2c P 998381aa0f734eec9531d0ce425fbe96: No bootstrap required, opened a new log
I20260806 22:49:52.691068 31219 ts_tablet_manager.cc:1397] T 8f3870fb45db42dcbdfc273e8d735e2c P 998381aa0f734eec9531d0ce425fbe96: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:49:52.691203 31221 log.cc:824] T 8f3870fb45db42dcbdfc273e8d735e2c P 36075fa9435445349d1c8429845ed592: Log is configured to *not* fsync() on all Append() calls
I20260806 22:49:52.691826 31221 tablet_bootstrap.cc:492] T 8f3870fb45db42dcbdfc273e8d735e2c P 36075fa9435445349d1c8429845ed592: No bootstrap required, opened a new log
I20260806 22:49:52.691905 31221 ts_tablet_manager.cc:1397] T 8f3870fb45db42dcbdfc273e8d735e2c P 36075fa9435445349d1c8429845ed592: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:49:52.692752 31220 raft_consensus.cc:348] T 8f3870fb45db42dcbdfc273e8d735e2c P f08aecdfef264c8b81b70ff01cbed9cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998381aa0f734eec9531d0ce425fbe96" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 45779 } } peers { permanent_uuid: "f08aecdfef264c8b81b70ff01cbed9cd" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 45763 } } peers { permanent_uuid: "36075fa9435445349d1c8429845ed592" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 34297 } }
I20260806 22:49:52.692919 31220 raft_consensus.cc:374] T 8f3870fb45db42dcbdfc273e8d735e2c P f08aecdfef264c8b81b70ff01cbed9cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:49:52.692955 31220 raft_consensus.cc:729] T 8f3870fb45db42dcbdfc273e8d735e2c P f08aecdfef264c8b81b70ff01cbed9cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f08aecdfef264c8b81b70ff01cbed9cd, State: Initialized, Role: FOLLOWER
I20260806 22:49:52.693063 31219 raft_consensus.cc:348] T 8f3870fb45db42dcbdfc273e8d735e2c P 998381aa0f734eec9531d0ce425fbe96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998381aa0f734eec9531d0ce425fbe96" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 45779 } } peers { permanent_uuid: "f08aecdfef264c8b81b70ff01cbed9cd" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 45763 } } peers { permanent_uuid: "36075fa9435445349d1c8429845ed592" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 34297 } }
I20260806 22:49:52.693133 31220 consensus_queue.cc:260] T 8f3870fb45db42dcbdfc273e8d735e2c P f08aecdfef264c8b81b70ff01cbed9cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998381aa0f734eec9531d0ce425fbe96" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 45779 } } peers { permanent_uuid: "f08aecdfef264c8b81b70ff01cbed9cd" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 45763 } } peers { permanent_uuid: "36075fa9435445349d1c8429845ed592" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 34297 } }
I20260806 22:49:52.693207 31219 raft_consensus.cc:374] T 8f3870fb45db42dcbdfc273e8d735e2c P 998381aa0f734eec9531d0ce425fbe96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:49:52.693246 31219 raft_consensus.cc:729] T 8f3870fb45db42dcbdfc273e8d735e2c P 998381aa0f734eec9531d0ce425fbe96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 998381aa0f734eec9531d0ce425fbe96, State: Initialized, Role: FOLLOWER
I20260806 22:49:52.693380 31220 ts_tablet_manager.cc:1428] T 8f3870fb45db42dcbdfc273e8d735e2c P f08aecdfef264c8b81b70ff01cbed9cd: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 22:49:52.693392 31219 consensus_queue.cc:260] T 8f3870fb45db42dcbdfc273e8d735e2c P 998381aa0f734eec9531d0ce425fbe96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998381aa0f734eec9531d0ce425fbe96" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 45779 } } peers { permanent_uuid: "f08aecdfef264c8b81b70ff01cbed9cd" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 45763 } } peers { permanent_uuid: "36075fa9435445349d1c8429845ed592" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 34297 } }
I20260806 22:49:52.693517 30905 heartbeater.cc:503] Master 127.28.218.254:43095 was elected leader, sending a full tablet report...
I20260806 22:49:52.693596 31221 raft_consensus.cc:348] T 8f3870fb45db42dcbdfc273e8d735e2c P 36075fa9435445349d1c8429845ed592 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998381aa0f734eec9531d0ce425fbe96" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 45779 } } peers { permanent_uuid: "f08aecdfef264c8b81b70ff01cbed9cd" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 45763 } } peers { permanent_uuid: "36075fa9435445349d1c8429845ed592" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 34297 } }
I20260806 22:49:52.693727 31221 raft_consensus.cc:374] T 8f3870fb45db42dcbdfc273e8d735e2c P 36075fa9435445349d1c8429845ed592 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:49:52.693763 31221 raft_consensus.cc:729] T 8f3870fb45db42dcbdfc273e8d735e2c P 36075fa9435445349d1c8429845ed592 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36075fa9435445349d1c8429845ed592, State: Initialized, Role: FOLLOWER
I20260806 22:49:52.693758 31196 heartbeater.cc:503] Master 127.28.218.254:43095 was elected leader, sending a full tablet report...
I20260806 22:49:52.693879 31221 consensus_queue.cc:260] T 8f3870fb45db42dcbdfc273e8d735e2c P 36075fa9435445349d1c8429845ed592 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998381aa0f734eec9531d0ce425fbe96" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 45779 } } peers { permanent_uuid: "f08aecdfef264c8b81b70ff01cbed9cd" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 45763 } } peers { permanent_uuid: "36075fa9435445349d1c8429845ed592" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 34297 } }
I20260806 22:49:52.693745 31219 ts_tablet_manager.cc:1428] T 8f3870fb45db42dcbdfc273e8d735e2c P 998381aa0f734eec9531d0ce425fbe96: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 22:49:52.694150 31221 ts_tablet_manager.cc:1428] T 8f3870fb45db42dcbdfc273e8d735e2c P 36075fa9435445349d1c8429845ed592: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:49:52.694293 31050 heartbeater.cc:503] Master 127.28.218.254:43095 was elected leader, sending a full tablet report...
W20260806 22:49:52.742619 31051 tablet.cc:2369] T 8f3870fb45db42dcbdfc273e8d735e2c P 36075fa9435445349d1c8429845ed592: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 22:49:52.833623 30906 tablet.cc:2369] T 8f3870fb45db42dcbdfc273e8d735e2c P f08aecdfef264c8b81b70ff01cbed9cd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:49:52.844687 31226 raft_consensus.cc:482] T 8f3870fb45db42dcbdfc273e8d735e2c P 998381aa0f734eec9531d0ce425fbe96 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:49:52.844796 31226 raft_consensus.cc:504] T 8f3870fb45db42dcbdfc273e8d735e2c P 998381aa0f734eec9531d0ce425fbe96 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998381aa0f734eec9531d0ce425fbe96" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 45779 } } peers { permanent_uuid: "f08aecdfef264c8b81b70ff01cbed9cd" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 45763 } } peers { permanent_uuid: "36075fa9435445349d1c8429845ed592" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 34297 } }
I20260806 22:49:52.845139 31226 leader_election.cc:288] T 8f3870fb45db42dcbdfc273e8d735e2c P 998381aa0f734eec9531d0ce425fbe96 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f08aecdfef264c8b81b70ff01cbed9cd (127.28.218.193:45763), 36075fa9435445349d1c8429845ed592 (127.28.218.194:34297)
I20260806 22:49:52.848261 31215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:52.845338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.195:56253 (local address 127.28.218.193:45763)
0806 22:49:52.845382 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:52.845385 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:52.845471 (+ 86us) server_negotiation.cc:407] Connection header received
0806 22:49:52.845503 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:52.845504 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:52.845514 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:52.845603 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:49:52.845786 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:52.846339 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:52.847479 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:52.847883 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:52.847998 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:52.848006 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:52.848154 (+ 148us) server_negotiation.cc:299] Negotiation successful
0806 22:49:52.848191 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 22:49:52.848276 31218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:52.845399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.195:56119 (local address 127.28.218.194:34297)
0806 22:49:52.845449 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:52.845451 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:52.845715 (+ 264us) server_negotiation.cc:407] Connection header received
0806 22:49:52.845744 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:52.845745 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:52.845756 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:52.845819 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:49:52.845968 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:52.846434 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:52.847524 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:52.847892 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:52.848002 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:52.848012 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:52.848172 (+ 160us) server_negotiation.cc:299] Negotiation successful
0806 22:49:52.848215 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 22:49:52.848266 31085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:52.845306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.193:45763 (local address 127.28.218.195:56253)
0806 22:49:52.845362 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:52.845374 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:52.845435 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:52.845622 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:52.845623 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:52.845624 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:49:52.845720 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:52.845722 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:52.846424 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:52.846425 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:52.847385 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:52.847387 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:52.847994 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:52.847995 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:52.848077 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:52.848081 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:49:52.848123 (+ 42us) client_negotiation.cc:240] Negotiation successful
0806 22:49:52.848181 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33,"mutex_wait_us":43}
I20260806 22:49:52.848284 31228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:52.845312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.194:34297 (local address 127.28.218.195:56119)
0806 22:49:52.845644 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:52.845653 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:52.845698 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:52.845832 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:52.845833 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:52.845834 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:49:52.845903 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:52.845904 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:52.846512 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:52.846512 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:52.847396 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:52.847399 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:52.848001 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:52.848002 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:52.848083 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:52.848086 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:49:52.848156 (+ 70us) client_negotiation.cc:240] Negotiation successful
0806 22:49:52.848196 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":311,"mutex_wait_us":181,"thread_start_us":98,"threads_started":1}
I20260806 22:49:52.848631 30849 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f3870fb45db42dcbdfc273e8d735e2c" candidate_uuid: "998381aa0f734eec9531d0ce425fbe96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f08aecdfef264c8b81b70ff01cbed9cd" is_pre_election: true
I20260806 22:49:52.848752 30849 raft_consensus.cc:2454] T 8f3870fb45db42dcbdfc273e8d735e2c P f08aecdfef264c8b81b70ff01cbed9cd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 998381aa0f734eec9531d0ce425fbe96 in term 0.
I20260806 22:49:52.848958 31078 leader_election.cc:302] T 8f3870fb45db42dcbdfc273e8d735e2c P 998381aa0f734eec9531d0ce425fbe96 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 998381aa0f734eec9531d0ce425fbe96, f08aecdfef264c8b81b70ff01cbed9cd; no voters:
I20260806 22:49:52.848956 31003 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f3870fb45db42dcbdfc273e8d735e2c" candidate_uuid: "998381aa0f734eec9531d0ce425fbe96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36075fa9435445349d1c8429845ed592" is_pre_election: true
I20260806 22:49:52.849089 31003 raft_consensus.cc:2454] T 8f3870fb45db42dcbdfc273e8d735e2c P 36075fa9435445349d1c8429845ed592 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 998381aa0f734eec9531d0ce425fbe96 in term 0.
I20260806 22:49:52.849104 31226 raft_consensus.cc:2781] T 8f3870fb45db42dcbdfc273e8d735e2c P 998381aa0f734eec9531d0ce425fbe96 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:49:52.849159 31226 raft_consensus.cc:482] T 8f3870fb45db42dcbdfc273e8d735e2c P 998381aa0f734eec9531d0ce425fbe96 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:49:52.849210 31226 raft_consensus.cc:3037] T 8f3870fb45db42dcbdfc273e8d735e2c P 998381aa0f734eec9531d0ce425fbe96 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:49:52.850169 31226 raft_consensus.cc:504] T 8f3870fb45db42dcbdfc273e8d735e2c P 998381aa0f734eec9531d0ce425fbe96 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998381aa0f734eec9531d0ce425fbe96" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 45779 } } peers { permanent_uuid: "f08aecdfef264c8b81b70ff01cbed9cd" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 45763 } } peers { permanent_uuid: "36075fa9435445349d1c8429845ed592" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 34297 } }
I20260806 22:49:52.850307 31226 leader_election.cc:288] T 8f3870fb45db42dcbdfc273e8d735e2c P 998381aa0f734eec9531d0ce425fbe96 [CANDIDATE]: Term 1 election: Requested vote from peers f08aecdfef264c8b81b70ff01cbed9cd (127.28.218.193:45763), 36075fa9435445349d1c8429845ed592 (127.28.218.194:34297)
I20260806 22:49:52.850467 30849 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f3870fb45db42dcbdfc273e8d735e2c" candidate_uuid: "998381aa0f734eec9531d0ce425fbe96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f08aecdfef264c8b81b70ff01cbed9cd"
I20260806 22:49:52.850530 30849 raft_consensus.cc:3037] T 8f3870fb45db42dcbdfc273e8d735e2c P f08aecdfef264c8b81b70ff01cbed9cd [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:49:52.851183 31003 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f3870fb45db42dcbdfc273e8d735e2c" candidate_uuid: "998381aa0f734eec9531d0ce425fbe96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36075fa9435445349d1c8429845ed592"
I20260806 22:49:52.851208 30849 raft_consensus.cc:2454] T 8f3870fb45db42dcbdfc273e8d735e2c P f08aecdfef264c8b81b70ff01cbed9cd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 998381aa0f734eec9531d0ce425fbe96 in term 1.
I20260806 22:49:52.851267 31003 raft_consensus.cc:3037] T 8f3870fb45db42dcbdfc273e8d735e2c P 36075fa9435445349d1c8429845ed592 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:49:52.851369 31078 leader_election.cc:302] T 8f3870fb45db42dcbdfc273e8d735e2c P 998381aa0f734eec9531d0ce425fbe96 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 998381aa0f734eec9531d0ce425fbe96, f08aecdfef264c8b81b70ff01cbed9cd; no voters:
I20260806 22:49:52.851469 31226 raft_consensus.cc:2781] T 8f3870fb45db42dcbdfc273e8d735e2c P 998381aa0f734eec9531d0ce425fbe96 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:49:52.851626 31226 raft_consensus.cc:686] T 8f3870fb45db42dcbdfc273e8d735e2c P 998381aa0f734eec9531d0ce425fbe96 [term 1 LEADER]: Becoming Leader. State: Replica: 998381aa0f734eec9531d0ce425fbe96, State: Running, Role: LEADER
I20260806 22:49:52.851715 31226 consensus_queue.cc:237] T 8f3870fb45db42dcbdfc273e8d735e2c P 998381aa0f734eec9531d0ce425fbe96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998381aa0f734eec9531d0ce425fbe96" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 45779 } } peers { permanent_uuid: "f08aecdfef264c8b81b70ff01cbed9cd" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 45763 } } peers { permanent_uuid: "36075fa9435445349d1c8429845ed592" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 34297 } }
I20260806 22:49:52.852154 31003 raft_consensus.cc:2454] T 8f3870fb45db42dcbdfc273e8d735e2c P 36075fa9435445349d1c8429845ed592 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 998381aa0f734eec9531d0ce425fbe96 in term 1.
I20260806 22:49:52.852537 30693 catalog_manager.cc:5495] T 8f3870fb45db42dcbdfc273e8d735e2c P 998381aa0f734eec9531d0ce425fbe96 reported cstate change: term changed from 0 to 1, leader changed from <none> to 998381aa0f734eec9531d0ce425fbe96 (127.28.218.195). New cstate: current_term: 1 leader_uuid: "998381aa0f734eec9531d0ce425fbe96" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998381aa0f734eec9531d0ce425fbe96" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 45779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f08aecdfef264c8b81b70ff01cbed9cd" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 45763 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "36075fa9435445349d1c8429845ed592" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 34297 } health_report { overall_health: UNKNOWN } } }
I20260806 22:49:52.863730 31217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:52.861484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55394 (local address 127.28.218.195:45779)
0806 22:49:52.861553 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:52.861554 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:52.861563 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:49:52.861632 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:52.861632 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:52.861645 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:52.861700 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:49:52.861820 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:52.862249 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:52.862931 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:52.863101 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:52.863207 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:52.863391 (+ 184us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:49:52.863483 (+ 92us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:49:52.863564 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:52.863616 (+ 52us) server_negotiation.cc:299] Negotiation successful
0806 22:49:52.863634 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 22:49:52.865799 31003 raft_consensus.cc:1261] T 8f3870fb45db42dcbdfc273e8d735e2c P 36075fa9435445349d1c8429845ed592 [term 1 FOLLOWER]: Refusing update from remote peer 998381aa0f734eec9531d0ce425fbe96: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:49:52.865942 30849 raft_consensus.cc:1261] T 8f3870fb45db42dcbdfc273e8d735e2c P f08aecdfef264c8b81b70ff01cbed9cd [term 1 FOLLOWER]: Refusing update from remote peer 998381aa0f734eec9531d0ce425fbe96: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:49:52.866151 31226 consensus_queue.cc:1035] T 8f3870fb45db42dcbdfc273e8d735e2c P 998381aa0f734eec9531d0ce425fbe96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "36075fa9435445349d1c8429845ed592" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 34297 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:49:52.866384 31226 consensus_queue.cc:1035] T 8f3870fb45db42dcbdfc273e8d735e2c P 998381aa0f734eec9531d0ce425fbe96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f08aecdfef264c8b81b70ff01cbed9cd" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 45763 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:49:52.869180 31232 mvcc.cc:205] Tried to move back new op lower bound from 7315687804375617536 to 7315687804321292288. Current Snapshot: MvccSnapshot[applied={T|T < 7315687804375617536 or (T in {7315687804375617536})}]
I20260806 22:49:52.869446 31234 mvcc.cc:205] Tried to move back new op lower bound from 7315687804375617536 to 7315687804321292288. Current Snapshot: MvccSnapshot[applied={T|T < 7315687804375617536 or (T in {7315687804375617536})}]
I20260806 22:49:52.871867 30693 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45642:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:49:52.871966 30693 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45642:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:49:52.872807 30693 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5e2fa0b04c9a4d41a3a5532eb98e4777: Sending DeleteTablet for 3 replicas of tablet 8f3870fb45db42dcbdfc273e8d735e2c
I20260806 22:49:52.873267 30837 tablet_service.cc:1514] Processing DeleteTablet for tablet 8f3870fb45db42dcbdfc273e8d735e2c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:49:52 UTC) from {username='kudu', principal='kudu/127.28.218.254@KRBTEST.COM'} at 127.0.0.1:47754
I20260806 22:49:52.873440 31242 tablet_replica.cc:330] T 8f3870fb45db42dcbdfc273e8d735e2c P f08aecdfef264c8b81b70ff01cbed9cd: stopping tablet replica
I20260806 22:49:52.873515 31242 raft_consensus.cc:2229] T 8f3870fb45db42dcbdfc273e8d735e2c P f08aecdfef264c8b81b70ff01cbed9cd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:49:52.873585 31242 raft_consensus.cc:2258] T 8f3870fb45db42dcbdfc273e8d735e2c P f08aecdfef264c8b81b70ff01cbed9cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:49:52.873888 31242 ts_tablet_manager.cc:1905] T 8f3870fb45db42dcbdfc273e8d735e2c P f08aecdfef264c8b81b70ff01cbed9cd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:49:52.875278 31242 ts_tablet_manager.cc:1918] T 8f3870fb45db42dcbdfc273e8d735e2c P f08aecdfef264c8b81b70ff01cbed9cd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:49:52.875339 31242 log.cc:1196] T 8f3870fb45db42dcbdfc273e8d735e2c P f08aecdfef264c8b81b70ff01cbed9cd: Deleting WAL directory at /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056586788339-29547-0/minicluster-data/ts-0/wal/wals/8f3870fb45db42dcbdfc273e8d735e2c
I20260806 22:49:52.875607 31242 ts_tablet_manager.cc:1939] T 8f3870fb45db42dcbdfc273e8d735e2c P f08aecdfef264c8b81b70ff01cbed9cd: Deleting consensus metadata
I20260806 22:49:52.875674 30981 tablet_service.cc:1514] Processing DeleteTablet for tablet 8f3870fb45db42dcbdfc273e8d735e2c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:49:52 UTC) from {username='kudu', principal='kudu/127.28.218.254@KRBTEST.COM'} at 127.0.0.1:52420
I20260806 22:49:52.875903 31243 tablet_replica.cc:330] T 8f3870fb45db42dcbdfc273e8d735e2c P 36075fa9435445349d1c8429845ed592: stopping tablet replica
I20260806 22:49:52.875911 31127 tablet_service.cc:1514] Processing DeleteTablet for tablet 8f3870fb45db42dcbdfc273e8d735e2c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:49:52 UTC) from {username='kudu', principal='kudu/127.28.218.254@KRBTEST.COM'} at 127.0.0.1:55384
I20260806 22:49:52.876157 31244 tablet_replica.cc:330] T 8f3870fb45db42dcbdfc273e8d735e2c P 998381aa0f734eec9531d0ce425fbe96: stopping tablet replica
I20260806 22:49:52.876173 31243 raft_consensus.cc:2229] T 8f3870fb45db42dcbdfc273e8d735e2c P 36075fa9435445349d1c8429845ed592 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:49:52.876133 30679 catalog_manager.cc:4841] TS f08aecdfef264c8b81b70ff01cbed9cd (127.28.218.193:45763): tablet 8f3870fb45db42dcbdfc273e8d735e2c (table test-table [id=af3eaf639ff94e8798763c79de537bb5]) successfully deleted
I20260806 22:49:52.876308 31244 raft_consensus.cc:2229] T 8f3870fb45db42dcbdfc273e8d735e2c P 998381aa0f734eec9531d0ce425fbe96 [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:49:52.876314 31243 raft_consensus.cc:2258] T 8f3870fb45db42dcbdfc273e8d735e2c P 36075fa9435445349d1c8429845ed592 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:49:52.876295 30693 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45642:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 22:49:52.876441 31244 raft_consensus.cc:2258] T 8f3870fb45db42dcbdfc273e8d735e2c P 998381aa0f734eec9531d0ce425fbe96 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260806 22:49:52.876447 30693 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:49:52.876811 31243 ts_tablet_manager.cc:1905] T 8f3870fb45db42dcbdfc273e8d735e2c P 36075fa9435445349d1c8429845ed592: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:49:52.877702 30981 tablet_service.cc:1467] Processing CreateTablet for tablet dd6ee574fe5948219ec42c723dd5d263 (DEFAULT_TABLE table=test-table [id=8abe36678bf247f7b0cc42691b0f5956]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:49:52.877830 30981 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dd6ee574fe5948219ec42c723dd5d263. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:49:52.877821 30837 tablet_service.cc:1467] Processing CreateTablet for tablet dd6ee574fe5948219ec42c723dd5d263 (DEFAULT_TABLE table=test-table [id=8abe36678bf247f7b0cc42691b0f5956]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:49:52.877907 30837 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dd6ee574fe5948219ec42c723dd5d263. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:49:52.878381 31243 ts_tablet_manager.cc:1918] T 8f3870fb45db42dcbdfc273e8d735e2c P 36075fa9435445349d1c8429845ed592: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:49:52.878440 31243 log.cc:1196] T 8f3870fb45db42dcbdfc273e8d735e2c P 36075fa9435445349d1c8429845ed592: Deleting WAL directory at /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056586788339-29547-0/minicluster-data/ts-1/wal/wals/8f3870fb45db42dcbdfc273e8d735e2c
I20260806 22:49:52.878691 31243 ts_tablet_manager.cc:1939] T 8f3870fb45db42dcbdfc273e8d735e2c P 36075fa9435445349d1c8429845ed592: Deleting consensus metadata
I20260806 22:49:52.879046 31220 tablet_bootstrap.cc:492] T dd6ee574fe5948219ec42c723dd5d263 P f08aecdfef264c8b81b70ff01cbed9cd: Bootstrap starting.
I20260806 22:49:52.879341 31127 tablet_service.cc:1467] Processing CreateTablet for tablet dd6ee574fe5948219ec42c723dd5d263 (DEFAULT_TABLE table=test-table [id=8abe36678bf247f7b0cc42691b0f5956]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:49:52.879472 31127 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dd6ee574fe5948219ec42c723dd5d263. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:49:52.879489 31220 tablet_bootstrap.cc:654] T dd6ee574fe5948219ec42c723dd5d263 P f08aecdfef264c8b81b70ff01cbed9cd: Neither blocks nor log segments found. Creating new log.
I20260806 22:49:52.879812 31221 tablet_bootstrap.cc:492] T dd6ee574fe5948219ec42c723dd5d263 P 36075fa9435445349d1c8429845ed592: Bootstrap starting.
I20260806 22:49:52.880173 31221 tablet_bootstrap.cc:654] T dd6ee574fe5948219ec42c723dd5d263 P 36075fa9435445349d1c8429845ed592: Neither blocks nor log segments found. Creating new log.
I20260806 22:49:52.880630 31220 tablet_bootstrap.cc:492] T dd6ee574fe5948219ec42c723dd5d263 P f08aecdfef264c8b81b70ff01cbed9cd: No bootstrap required, opened a new log
I20260806 22:49:52.880661 31220 ts_tablet_manager.cc:1397] T dd6ee574fe5948219ec42c723dd5d263 P f08aecdfef264c8b81b70ff01cbed9cd: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:49:52.880779 31219 tablet_bootstrap.cc:492] T dd6ee574fe5948219ec42c723dd5d263 P 998381aa0f734eec9531d0ce425fbe96: Bootstrap starting.
I20260806 22:49:52.880762 31220 raft_consensus.cc:348] T dd6ee574fe5948219ec42c723dd5d263 P f08aecdfef264c8b81b70ff01cbed9cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f08aecdfef264c8b81b70ff01cbed9cd" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 45763 } } peers { permanent_uuid: "36075fa9435445349d1c8429845ed592" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 34297 } } peers { permanent_uuid: "998381aa0f734eec9531d0ce425fbe96" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 45779 } }
I20260806 22:49:52.880827 30678 catalog_manager.cc:4841] TS 36075fa9435445349d1c8429845ed592 (127.28.218.194:34297): tablet 8f3870fb45db42dcbdfc273e8d735e2c (table test-table [id=af3eaf639ff94e8798763c79de537bb5]) successfully deleted
I20260806 22:49:52.880844 31220 raft_consensus.cc:374] T dd6ee574fe5948219ec42c723dd5d263 P f08aecdfef264c8b81b70ff01cbed9cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:49:52.880882 31220 raft_consensus.cc:729] T dd6ee574fe5948219ec42c723dd5d263 P f08aecdfef264c8b81b70ff01cbed9cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f08aecdfef264c8b81b70ff01cbed9cd, State: Initialized, Role: FOLLOWER
I20260806 22:49:52.880946 31220 consensus_queue.cc:260] T dd6ee574fe5948219ec42c723dd5d263 P f08aecdfef264c8b81b70ff01cbed9cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f08aecdfef264c8b81b70ff01cbed9cd" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 45763 } } peers { permanent_uuid: "36075fa9435445349d1c8429845ed592" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 34297 } } peers { permanent_uuid: "998381aa0f734eec9531d0ce425fbe96" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 45779 } }
I20260806 22:49:52.881070 31220 ts_tablet_manager.cc:1428] T dd6ee574fe5948219ec42c723dd5d263 P f08aecdfef264c8b81b70ff01cbed9cd: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:52.881240 31219 tablet_bootstrap.cc:654] T dd6ee574fe5948219ec42c723dd5d263 P 998381aa0f734eec9531d0ce425fbe96: Neither blocks nor log segments found. Creating new log.
I20260806 22:49:52.881417 31221 tablet_bootstrap.cc:492] T dd6ee574fe5948219ec42c723dd5d263 P 36075fa9435445349d1c8429845ed592: No bootstrap required, opened a new log
I20260806 22:49:52.881446 31244 ts_tablet_manager.cc:1905] T 8f3870fb45db42dcbdfc273e8d735e2c P 998381aa0f734eec9531d0ce425fbe96: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:49:52.881461 31221 ts_tablet_manager.cc:1397] T dd6ee574fe5948219ec42c723dd5d263 P 36075fa9435445349d1c8429845ed592: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:49:52.881647 31221 raft_consensus.cc:348] T dd6ee574fe5948219ec42c723dd5d263 P 36075fa9435445349d1c8429845ed592 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f08aecdfef264c8b81b70ff01cbed9cd" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 45763 } } peers { permanent_uuid: "36075fa9435445349d1c8429845ed592" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 34297 } } peers { permanent_uuid: "998381aa0f734eec9531d0ce425fbe96" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 45779 } }
I20260806 22:49:52.881731 31221 raft_consensus.cc:374] T dd6ee574fe5948219ec42c723dd5d263 P 36075fa9435445349d1c8429845ed592 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:49:52.881753 31221 raft_consensus.cc:729] T dd6ee574fe5948219ec42c723dd5d263 P 36075fa9435445349d1c8429845ed592 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36075fa9435445349d1c8429845ed592, State: Initialized, Role: FOLLOWER
I20260806 22:49:52.881793 31221 consensus_queue.cc:260] T dd6ee574fe5948219ec42c723dd5d263 P 36075fa9435445349d1c8429845ed592 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f08aecdfef264c8b81b70ff01cbed9cd" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 45763 } } peers { permanent_uuid: "36075fa9435445349d1c8429845ed592" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 34297 } } peers { permanent_uuid: "998381aa0f734eec9531d0ce425fbe96" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 45779 } }
I20260806 22:49:52.881883 31221 ts_tablet_manager.cc:1428] T dd6ee574fe5948219ec42c723dd5d263 P 36075fa9435445349d1c8429845ed592: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260806 22:49:52.882532 31219 tablet_bootstrap.cc:492] T dd6ee574fe5948219ec42c723dd5d263 P 998381aa0f734eec9531d0ce425fbe96: No bootstrap required, opened a new log
I20260806 22:49:52.882597 31219 ts_tablet_manager.cc:1397] T dd6ee574fe5948219ec42c723dd5d263 P 998381aa0f734eec9531d0ce425fbe96: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:49:52.882750 31219 raft_consensus.cc:348] T dd6ee574fe5948219ec42c723dd5d263 P 998381aa0f734eec9531d0ce425fbe96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f08aecdfef264c8b81b70ff01cbed9cd" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 45763 } } peers { permanent_uuid: "36075fa9435445349d1c8429845ed592" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 34297 } } peers { permanent_uuid: "998381aa0f734eec9531d0ce425fbe96" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 45779 } }
I20260806 22:49:52.882799 31219 raft_consensus.cc:374] T dd6ee574fe5948219ec42c723dd5d263 P 998381aa0f734eec9531d0ce425fbe96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:49:52.882813 31219 raft_consensus.cc:729] T dd6ee574fe5948219ec42c723dd5d263 P 998381aa0f734eec9531d0ce425fbe96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 998381aa0f734eec9531d0ce425fbe96, State: Initialized, Role: FOLLOWER
I20260806 22:49:52.882871 31219 consensus_queue.cc:260] T dd6ee574fe5948219ec42c723dd5d263 P 998381aa0f734eec9531d0ce425fbe96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f08aecdfef264c8b81b70ff01cbed9cd" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 45763 } } peers { permanent_uuid: "36075fa9435445349d1c8429845ed592" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 34297 } } peers { permanent_uuid: "998381aa0f734eec9531d0ce425fbe96" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 45779 } }
I20260806 22:49:52.882922 31244 ts_tablet_manager.cc:1918] T 8f3870fb45db42dcbdfc273e8d735e2c P 998381aa0f734eec9531d0ce425fbe96: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:49:52.882973 31219 ts_tablet_manager.cc:1428] T dd6ee574fe5948219ec42c723dd5d263 P 998381aa0f734eec9531d0ce425fbe96: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:52.882990 31244 log.cc:1196] T 8f3870fb45db42dcbdfc273e8d735e2c P 998381aa0f734eec9531d0ce425fbe96: Deleting WAL directory at /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056586788339-29547-0/minicluster-data/ts-2/wal/wals/8f3870fb45db42dcbdfc273e8d735e2c
I20260806 22:49:52.883307 31244 ts_tablet_manager.cc:1939] T 8f3870fb45db42dcbdfc273e8d735e2c P 998381aa0f734eec9531d0ce425fbe96: Deleting consensus metadata
I20260806 22:49:52.883843 30678 catalog_manager.cc:4841] TS 998381aa0f734eec9531d0ce425fbe96 (127.28.218.195:45779): tablet 8f3870fb45db42dcbdfc273e8d735e2c (table test-table [id=af3eaf639ff94e8798763c79de537bb5]) successfully deleted
W20260806 22:49:52.895957 31197 tablet.cc:2369] T dd6ee574fe5948219ec42c723dd5d263 P 998381aa0f734eec9531d0ce425fbe96: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:49:53.107904 31225 raft_consensus.cc:482] T dd6ee574fe5948219ec42c723dd5d263 P f08aecdfef264c8b81b70ff01cbed9cd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:49:53.108019 31225 raft_consensus.cc:504] T dd6ee574fe5948219ec42c723dd5d263 P f08aecdfef264c8b81b70ff01cbed9cd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f08aecdfef264c8b81b70ff01cbed9cd" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 45763 } } peers { permanent_uuid: "36075fa9435445349d1c8429845ed592" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 34297 } } peers { permanent_uuid: "998381aa0f734eec9531d0ce425fbe96" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 45779 } }
I20260806 22:49:53.108350 31225 leader_election.cc:288] T dd6ee574fe5948219ec42c723dd5d263 P f08aecdfef264c8b81b70ff01cbed9cd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 36075fa9435445349d1c8429845ed592 (127.28.218.194:34297), 998381aa0f734eec9531d0ce425fbe96 (127.28.218.195:45779)
I20260806 22:49:53.111299 31218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:53.108604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.193:43425 (local address 127.28.218.194:34297)
0806 22:49:53.108675 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:53.108678 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:53.108737 (+ 59us) server_negotiation.cc:407] Connection header received
0806 22:49:53.108818 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:53.108820 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:53.108829 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:53.108893 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:49:53.109136 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:53.109603 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:53.110537 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:53.110906 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:53.110990 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:53.111002 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:53.111187 (+ 185us) server_negotiation.cc:299] Negotiation successful
0806 22:49:53.111229 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 22:49:53.111323 31249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:53.108525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.194:34297 (local address 127.28.218.193:43425)
0806 22:49:53.108674 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:53.108685 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:53.108749 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:53.108966 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:53.108967 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:53.108969 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:49:53.109046 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:53.109048 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:53.109685 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:53.109686 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:53.110442 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:53.110444 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:53.111013 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:53.111015 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:53.111079 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:53.111082 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:49:53.111175 (+ 93us) client_negotiation.cc:240] Negotiation successful
0806 22:49:53.111214 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":71,"threads_started":1}
I20260806 22:49:53.111368 31217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:53.108834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.193:48371 (local address 127.28.218.195:45779)
0806 22:49:53.108917 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:53.108919 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:53.108929 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:49:53.108958 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:53.108959 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:53.108971 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:53.109042 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:49:53.109203 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:53.109643 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:53.110539 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:53.110914 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:53.110999 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:53.111007 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:53.111237 (+ 230us) server_negotiation.cc:299] Negotiation successful
0806 22:49:53.111276 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 22:49:53.111601 31250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:53.108709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.195:45779 (local address 127.28.218.193:48371)
0806 22:49:53.108856 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:53.108867 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:53.108907 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:53.109045 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:53.109046 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:53.109048 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:49:53.109135 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:53.109136 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:53.109725 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:53.109726 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:53.110445 (+ 719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:53.110447 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:53.111031 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:53.111032 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:53.111104 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:53.111108 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:49:53.111226 (+ 118us) client_negotiation.cc:240] Negotiation successful
0806 22:49:53.111266 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":86,"threads_started":1}
I20260806 22:49:53.111781 31003 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd6ee574fe5948219ec42c723dd5d263" candidate_uuid: "f08aecdfef264c8b81b70ff01cbed9cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36075fa9435445349d1c8429845ed592" is_pre_election: true
I20260806 22:49:53.111877 31003 raft_consensus.cc:2454] T dd6ee574fe5948219ec42c723dd5d263 P 36075fa9435445349d1c8429845ed592 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f08aecdfef264c8b81b70ff01cbed9cd in term 0.
I20260806 22:49:53.111836 31147 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd6ee574fe5948219ec42c723dd5d263" candidate_uuid: "f08aecdfef264c8b81b70ff01cbed9cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998381aa0f734eec9531d0ce425fbe96" is_pre_election: true
I20260806 22:49:53.111994 31147 raft_consensus.cc:2454] T dd6ee574fe5948219ec42c723dd5d263 P 998381aa0f734eec9531d0ce425fbe96 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f08aecdfef264c8b81b70ff01cbed9cd in term 0.
I20260806 22:49:53.112067 30787 leader_election.cc:302] T dd6ee574fe5948219ec42c723dd5d263 P f08aecdfef264c8b81b70ff01cbed9cd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36075fa9435445349d1c8429845ed592, f08aecdfef264c8b81b70ff01cbed9cd; no voters:
I20260806 22:49:53.112215 31225 raft_consensus.cc:2781] T dd6ee574fe5948219ec42c723dd5d263 P f08aecdfef264c8b81b70ff01cbed9cd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:49:53.112258 31225 raft_consensus.cc:482] T dd6ee574fe5948219ec42c723dd5d263 P f08aecdfef264c8b81b70ff01cbed9cd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:49:53.112285 31225 raft_consensus.cc:3037] T dd6ee574fe5948219ec42c723dd5d263 P f08aecdfef264c8b81b70ff01cbed9cd [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:49:53.112975 31225 raft_consensus.cc:504] T dd6ee574fe5948219ec42c723dd5d263 P f08aecdfef264c8b81b70ff01cbed9cd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f08aecdfef264c8b81b70ff01cbed9cd" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 45763 } } peers { permanent_uuid: "36075fa9435445349d1c8429845ed592" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 34297 } } peers { permanent_uuid: "998381aa0f734eec9531d0ce425fbe96" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 45779 } }
I20260806 22:49:53.113133 31225 leader_election.cc:288] T dd6ee574fe5948219ec42c723dd5d263 P f08aecdfef264c8b81b70ff01cbed9cd [CANDIDATE]: Term 1 election: Requested vote from peers 36075fa9435445349d1c8429845ed592 (127.28.218.194:34297), 998381aa0f734eec9531d0ce425fbe96 (127.28.218.195:45779)
I20260806 22:49:53.113279 31003 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd6ee574fe5948219ec42c723dd5d263" candidate_uuid: "f08aecdfef264c8b81b70ff01cbed9cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36075fa9435445349d1c8429845ed592"
I20260806 22:49:53.113307 31147 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd6ee574fe5948219ec42c723dd5d263" candidate_uuid: "f08aecdfef264c8b81b70ff01cbed9cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "998381aa0f734eec9531d0ce425fbe96"
I20260806 22:49:53.113350 31003 raft_consensus.cc:3037] T dd6ee574fe5948219ec42c723dd5d263 P 36075fa9435445349d1c8429845ed592 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:49:53.113384 31147 raft_consensus.cc:3037] T dd6ee574fe5948219ec42c723dd5d263 P 998381aa0f734eec9531d0ce425fbe96 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:49:53.114028 31003 raft_consensus.cc:2454] T dd6ee574fe5948219ec42c723dd5d263 P 36075fa9435445349d1c8429845ed592 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f08aecdfef264c8b81b70ff01cbed9cd in term 1.
I20260806 22:49:53.114089 31147 raft_consensus.cc:2454] T dd6ee574fe5948219ec42c723dd5d263 P 998381aa0f734eec9531d0ce425fbe96 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f08aecdfef264c8b81b70ff01cbed9cd in term 1.
I20260806 22:49:53.114214 30787 leader_election.cc:302] T dd6ee574fe5948219ec42c723dd5d263 P f08aecdfef264c8b81b70ff01cbed9cd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36075fa9435445349d1c8429845ed592, f08aecdfef264c8b81b70ff01cbed9cd; no voters:
I20260806 22:49:53.114336 31225 raft_consensus.cc:2781] T dd6ee574fe5948219ec42c723dd5d263 P f08aecdfef264c8b81b70ff01cbed9cd [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:49:53.114473 31225 raft_consensus.cc:686] T dd6ee574fe5948219ec42c723dd5d263 P f08aecdfef264c8b81b70ff01cbed9cd [term 1 LEADER]: Becoming Leader. State: Replica: f08aecdfef264c8b81b70ff01cbed9cd, State: Running, Role: LEADER
I20260806 22:49:53.114539 31225 consensus_queue.cc:237] T dd6ee574fe5948219ec42c723dd5d263 P f08aecdfef264c8b81b70ff01cbed9cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f08aecdfef264c8b81b70ff01cbed9cd" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 45763 } } peers { permanent_uuid: "36075fa9435445349d1c8429845ed592" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 34297 } } peers { permanent_uuid: "998381aa0f734eec9531d0ce425fbe96" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 45779 } }
I20260806 22:49:53.117336 30693 catalog_manager.cc:5495] T dd6ee574fe5948219ec42c723dd5d263 P f08aecdfef264c8b81b70ff01cbed9cd reported cstate change: term changed from 0 to 1, leader changed from <none> to f08aecdfef264c8b81b70ff01cbed9cd (127.28.218.193). New cstate: current_term: 1 leader_uuid: "f08aecdfef264c8b81b70ff01cbed9cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f08aecdfef264c8b81b70ff01cbed9cd" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 45763 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "36075fa9435445349d1c8429845ed592" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 34297 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "998381aa0f734eec9531d0ce425fbe96" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 45779 } health_report { overall_health: UNKNOWN } } }
Aug 06 22:49:53 dist-test-slave-1xrh krb5kdc[30645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056592, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.254@KRBTEST.COM for kudu/127.28.218.254@KRBTEST.COM
I20260806 22:49:53.181923 31258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:53.175626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:43095 (local address 127.0.0.1:45644)
0806 22:49:53.175769 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:53.175778 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:53.175818 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:53.175970 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:53.175971 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:53.176248 (+ 277us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:49:53.176290 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:53.176292 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:53.176905 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:53.176906 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:53.177418 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:53.177419 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:53.177729 (+ 310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:53.177729 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:53.177767 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:53.178218 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:49:53.178222 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:49:53.179588 (+ 1366us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:49:53.180963 (+ 1375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:53.180969 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:53.180971 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:53.181167 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:53.181339 (+ 172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:53.181340 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:53.181340 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:53.181395 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:53.181669 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:49:53.181675 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:49:53.181722 (+ 47us) client_negotiation.cc:769] Sending connection context
0806 22:49:53.181773 (+ 51us) client_negotiation.cc:240] Negotiation successful
0806 22:49:53.181849 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":60,"threads_started":1}
I20260806 22:49:53.182035 31259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:53.175707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45644 (local address 127.28.218.254:43095)
0806 22:49:53.175857 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:53.175859 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:53.175866 (+ 7us) server_negotiation.cc:407] Connection header received
0806 22:49:53.175884 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:53.175884 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:53.175893 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:53.175954 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:53.176357 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:53.176842 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:53.177487 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:53.177667 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:53.177714 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:53.179671 (+ 1957us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:53.179672 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:53.179673 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:49:53.179678 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:53.180895 (+ 1217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:53.181230 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:53.181230 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:53.181231 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:53.181283 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:53.181485 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:53.181486 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:53.181486 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:53.181603 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:53.181671 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:53.181843 (+ 172us) server_negotiation.cc:299] Negotiation successful
0806 22:49:53.181918 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":92,"threads_started":1}
I20260806 22:49:53.184049 30693 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.28.218.254@KRBTEST.COM'} at 127.0.0.1:45644:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 22:49:53.184317 30693 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:49:53.185811 30837 tablet_service.cc:1467] Processing CreateTablet for tablet 231ba73b124a4c2093b04fe8b6b6d9f1 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=bb1b603473014fe0a8b6dbd244cbe099]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 22:49:53.185804 31127 tablet_service.cc:1467] Processing CreateTablet for tablet 231ba73b124a4c2093b04fe8b6b6d9f1 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=bb1b603473014fe0a8b6dbd244cbe099]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 22:49:53.185966 30837 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 231ba73b124a4c2093b04fe8b6b6d9f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:49:53.185966 31127 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 231ba73b124a4c2093b04fe8b6b6d9f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:49:53.187578 31220 tablet_bootstrap.cc:492] T 231ba73b124a4c2093b04fe8b6b6d9f1 P f08aecdfef264c8b81b70ff01cbed9cd: Bootstrap starting.
I20260806 22:49:53.188019 31220 tablet_bootstrap.cc:654] T 231ba73b124a4c2093b04fe8b6b6d9f1 P f08aecdfef264c8b81b70ff01cbed9cd: Neither blocks nor log segments found. Creating new log.
I20260806 22:49:53.188756 31220 tablet_bootstrap.cc:492] T 231ba73b124a4c2093b04fe8b6b6d9f1 P f08aecdfef264c8b81b70ff01cbed9cd: No bootstrap required, opened a new log
I20260806 22:49:53.188812 31220 ts_tablet_manager.cc:1397] T 231ba73b124a4c2093b04fe8b6b6d9f1 P f08aecdfef264c8b81b70ff01cbed9cd: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 22:49:53.188984 31220 raft_consensus.cc:348] T 231ba73b124a4c2093b04fe8b6b6d9f1 P f08aecdfef264c8b81b70ff01cbed9cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f08aecdfef264c8b81b70ff01cbed9cd" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 45763 } } peers { permanent_uuid: "36075fa9435445349d1c8429845ed592" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 34297 } } peers { permanent_uuid: "998381aa0f734eec9531d0ce425fbe96" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 45779 } }
I20260806 22:49:53.189059 31220 raft_consensus.cc:374] T 231ba73b124a4c2093b04fe8b6b6d9f1 P f08aecdfef264c8b81b70ff01cbed9cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:49:53.189105 31220 raft_consensus.cc:729] T 231ba73b124a4c2093b04fe8b6b6d9f1 P f08aecdfef264c8b81b70ff01cbed9cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f08aecdfef264c8b81b70ff01cbed9cd, State: Initialized, Role: FOLLOWER
I20260806 22:49:53.189152 31220 consensus_queue.cc:260] T 231ba73b124a4c2093b04fe8b6b6d9f1 P f08aecdfef264c8b81b70ff01cbed9cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f08aecdfef264c8b81b70ff01cbed9cd" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 45763 } } peers { permanent_uuid: "36075fa9435445349d1c8429845ed592" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 34297 } } peers { permanent_uuid: "998381aa0f734eec9531d0ce425fbe96" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 45779 } }
I20260806 22:49:53.189240 31220 ts_tablet_manager.cc:1428] T 231ba73b124a4c2093b04fe8b6b6d9f1 P f08aecdfef264c8b81b70ff01cbed9cd: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260806 22:49:53.189347 31225 tablet_replica.cc:439] T 231ba73b124a4c2093b04fe8b6b6d9f1 P f08aecdfef264c8b81b70ff01cbed9cd: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f08aecdfef264c8b81b70ff01cbed9cd" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 45763 } } peers { permanent_uuid: "36075fa9435445349d1c8429845ed592" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 34297 } } peers { permanent_uuid: "998381aa0f734eec9531d0ce425fbe96" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 45779 } } }
I20260806 22:49:53.189404 31225 tablet_replica.cc:442] T 231ba73b124a4c2093b04fe8b6b6d9f1 P f08aecdfef264c8b81b70ff01cbed9cd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:49:53.189621 31219 tablet_bootstrap.cc:492] T 231ba73b124a4c2093b04fe8b6b6d9f1 P 998381aa0f734eec9531d0ce425fbe96: Bootstrap starting.
I20260806 22:49:53.189736 30981 tablet_service.cc:1467] Processing CreateTablet for tablet 231ba73b124a4c2093b04fe8b6b6d9f1 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=bb1b603473014fe0a8b6dbd244cbe099]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 22:49:53.189875 30981 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 231ba73b124a4c2093b04fe8b6b6d9f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:49:53.190026 31219 tablet_bootstrap.cc:654] T 231ba73b124a4c2093b04fe8b6b6d9f1 P 998381aa0f734eec9531d0ce425fbe96: Neither blocks nor log segments found. Creating new log.
I20260806 22:49:53.190585 31219 tablet_bootstrap.cc:492] T 231ba73b124a4c2093b04fe8b6b6d9f1 P 998381aa0f734eec9531d0ce425fbe96: No bootstrap required, opened a new log
I20260806 22:49:53.190644 31219 ts_tablet_manager.cc:1397] T 231ba73b124a4c2093b04fe8b6b6d9f1 P 998381aa0f734eec9531d0ce425fbe96: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 22:49:53.190757 31219 raft_consensus.cc:348] T 231ba73b124a4c2093b04fe8b6b6d9f1 P 998381aa0f734eec9531d0ce425fbe96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f08aecdfef264c8b81b70ff01cbed9cd" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 45763 } } peers { permanent_uuid: "36075fa9435445349d1c8429845ed592" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 34297 } } peers { permanent_uuid: "998381aa0f734eec9531d0ce425fbe96" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 45779 } }
I20260806 22:49:53.190874 31219 raft_consensus.cc:374] T 231ba73b124a4c2093b04fe8b6b6d9f1 P 998381aa0f734eec9531d0ce425fbe96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:49:53.190904 31219 raft_consensus.cc:729] T 231ba73b124a4c2093b04fe8b6b6d9f1 P 998381aa0f734eec9531d0ce425fbe96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 998381aa0f734eec9531d0ce425fbe96, State: Initialized, Role: FOLLOWER
I20260806 22:49:53.190955 31219 consensus_queue.cc:260] T 231ba73b124a4c2093b04fe8b6b6d9f1 P 998381aa0f734eec9531d0ce425fbe96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f08aecdfef264c8b81b70ff01cbed9cd" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 45763 } } peers { permanent_uuid: "36075fa9435445349d1c8429845ed592" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 34297 } } peers { permanent_uuid: "998381aa0f734eec9531d0ce425fbe96" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 45779 } }
I20260806 22:49:53.191047 31219 ts_tablet_manager.cc:1428] T 231ba73b124a4c2093b04fe8b6b6d9f1 P 998381aa0f734eec9531d0ce425fbe96: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260806 22:49:53.191203 31229 tablet_replica.cc:439] T 231ba73b124a4c2093b04fe8b6b6d9f1 P 998381aa0f734eec9531d0ce425fbe96: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f08aecdfef264c8b81b70ff01cbed9cd" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 45763 } } peers { permanent_uuid: "36075fa9435445349d1c8429845ed592" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 34297 } } peers { permanent_uuid: "998381aa0f734eec9531d0ce425fbe96" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 45779 } } }
I20260806 22:49:53.191291 31229 tablet_replica.cc:442] T 231ba73b124a4c2093b04fe8b6b6d9f1 P 998381aa0f734eec9531d0ce425fbe96: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:49:53.191516 31221 tablet_bootstrap.cc:492] T 231ba73b124a4c2093b04fe8b6b6d9f1 P 36075fa9435445349d1c8429845ed592: Bootstrap starting.
I20260806 22:49:53.191895 31221 tablet_bootstrap.cc:654] T 231ba73b124a4c2093b04fe8b6b6d9f1 P 36075fa9435445349d1c8429845ed592: Neither blocks nor log segments found. Creating new log.
I20260806 22:49:53.192459 31221 tablet_bootstrap.cc:492] T 231ba73b124a4c2093b04fe8b6b6d9f1 P 36075fa9435445349d1c8429845ed592: No bootstrap required, opened a new log
I20260806 22:49:53.192500 31221 ts_tablet_manager.cc:1397] T 231ba73b124a4c2093b04fe8b6b6d9f1 P 36075fa9435445349d1c8429845ed592: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 22:49:53.192656 31221 raft_consensus.cc:348] T 231ba73b124a4c2093b04fe8b6b6d9f1 P 36075fa9435445349d1c8429845ed592 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f08aecdfef264c8b81b70ff01cbed9cd" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 45763 } } peers { permanent_uuid: "36075fa9435445349d1c8429845ed592" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 34297 } } peers { permanent_uuid: "998381aa0f734eec9531d0ce425fbe96" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 45779 } }
I20260806 22:49:53.192708 31221 raft_consensus.cc:374] T 231ba73b124a4c2093b04fe8b6b6d9f1 P 36075fa9435445349d1c8429845ed592 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:49:53.192749 31221 raft_consensus.cc:729] T 231ba73b124a4c2093b04fe8b6b6d9f1 P 36075fa9435445349d1c8429845ed592 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36075fa9435445349d1c8429845ed592, State: Initialized, Role: FOLLOWER
I20260806 22:49:53.192800 31221 consensus_queue.cc:260] T 231ba73b124a4c2093b04fe8b6b6d9f1 P 36075fa9435445349d1c8429845ed592 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f08aecdfef264c8b81b70ff01cbed9cd" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 45763 } } peers { permanent_uuid: "36075fa9435445349d1c8429845ed592" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 34297 } } peers { permanent_uuid: "998381aa0f734eec9531d0ce425fbe96" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 45779 } }
I20260806 22:49:53.192890 31221 ts_tablet_manager.cc:1428] T 231ba73b124a4c2093b04fe8b6b6d9f1 P 36075fa9435445349d1c8429845ed592: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260806 22:49:53.193010 31227 tablet_replica.cc:439] T 231ba73b124a4c2093b04fe8b6b6d9f1 P 36075fa9435445349d1c8429845ed592: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f08aecdfef264c8b81b70ff01cbed9cd" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 45763 } } peers { permanent_uuid: "36075fa9435445349d1c8429845ed592" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 34297 } } peers { permanent_uuid: "998381aa0f734eec9531d0ce425fbe96" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 45779 } } }
I20260806 22:49:53.193073 31227 tablet_replica.cc:442] T 231ba73b124a4c2093b04fe8b6b6d9f1 P 36075fa9435445349d1c8429845ed592: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:49:53.215381 31229 raft_consensus.cc:482] T 231ba73b124a4c2093b04fe8b6b6d9f1 P 998381aa0f734eec9531d0ce425fbe96 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:49:53.215469 31229 raft_consensus.cc:504] T 231ba73b124a4c2093b04fe8b6b6d9f1 P 998381aa0f734eec9531d0ce425fbe96 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f08aecdfef264c8b81b70ff01cbed9cd" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 45763 } } peers { permanent_uuid: "36075fa9435445349d1c8429845ed592" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 34297 } } peers { permanent_uuid: "998381aa0f734eec9531d0ce425fbe96" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 45779 } }
I20260806 22:49:53.215643 31229 leader_election.cc:288] T 231ba73b124a4c2093b04fe8b6b6d9f1 P 998381aa0f734eec9531d0ce425fbe96 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f08aecdfef264c8b81b70ff01cbed9cd (127.28.218.193:45763), 36075fa9435445349d1c8429845ed592 (127.28.218.194:34297)
I20260806 22:49:53.215834 31003 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "231ba73b124a4c2093b04fe8b6b6d9f1" candidate_uuid: "998381aa0f734eec9531d0ce425fbe96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36075fa9435445349d1c8429845ed592" is_pre_election: true
I20260806 22:49:53.215857 30849 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "231ba73b124a4c2093b04fe8b6b6d9f1" candidate_uuid: "998381aa0f734eec9531d0ce425fbe96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f08aecdfef264c8b81b70ff01cbed9cd" is_pre_election: true
I20260806 22:49:53.215924 31003 raft_consensus.cc:2454] T 231ba73b124a4c2093b04fe8b6b6d9f1 P 36075fa9435445349d1c8429845ed592 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 998381aa0f734eec9531d0ce425fbe96 in term 0.
I20260806 22:49:53.215956 30849 raft_consensus.cc:2454] T 231ba73b124a4c2093b04fe8b6b6d9f1 P f08aecdfef264c8b81b70ff01cbed9cd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 998381aa0f734eec9531d0ce425fbe96 in term 0.
I20260806 22:49:53.216069 31077 leader_election.cc:302] T 231ba73b124a4c2093b04fe8b6b6d9f1 P 998381aa0f734eec9531d0ce425fbe96 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36075fa9435445349d1c8429845ed592, 998381aa0f734eec9531d0ce425fbe96; no voters:
I20260806 22:49:53.216176 31229 raft_consensus.cc:2781] T 231ba73b124a4c2093b04fe8b6b6d9f1 P 998381aa0f734eec9531d0ce425fbe96 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:49:53.216223 31229 raft_consensus.cc:482] T 231ba73b124a4c2093b04fe8b6b6d9f1 P 998381aa0f734eec9531d0ce425fbe96 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:49:53.216236 31229 raft_consensus.cc:3037] T 231ba73b124a4c2093b04fe8b6b6d9f1 P 998381aa0f734eec9531d0ce425fbe96 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:49:53.216848 31229 raft_consensus.cc:504] T 231ba73b124a4c2093b04fe8b6b6d9f1 P 998381aa0f734eec9531d0ce425fbe96 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f08aecdfef264c8b81b70ff01cbed9cd" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 45763 } } peers { permanent_uuid: "36075fa9435445349d1c8429845ed592" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 34297 } } peers { permanent_uuid: "998381aa0f734eec9531d0ce425fbe96" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 45779 } }
I20260806 22:49:53.216995 31229 leader_election.cc:288] T 231ba73b124a4c2093b04fe8b6b6d9f1 P 998381aa0f734eec9531d0ce425fbe96 [CANDIDATE]: Term 1 election: Requested vote from peers f08aecdfef264c8b81b70ff01cbed9cd (127.28.218.193:45763), 36075fa9435445349d1c8429845ed592 (127.28.218.194:34297)
I20260806 22:49:53.217109 30849 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "231ba73b124a4c2093b04fe8b6b6d9f1" candidate_uuid: "998381aa0f734eec9531d0ce425fbe96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f08aecdfef264c8b81b70ff01cbed9cd"
I20260806 22:49:53.217202 30849 raft_consensus.cc:3037] T 231ba73b124a4c2093b04fe8b6b6d9f1 P f08aecdfef264c8b81b70ff01cbed9cd [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:49:53.217175 31003 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "231ba73b124a4c2093b04fe8b6b6d9f1" candidate_uuid: "998381aa0f734eec9531d0ce425fbe96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36075fa9435445349d1c8429845ed592"
I20260806 22:49:53.217265 31003 raft_consensus.cc:3037] T 231ba73b124a4c2093b04fe8b6b6d9f1 P 36075fa9435445349d1c8429845ed592 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:49:53.217787 30849 raft_consensus.cc:2454] T 231ba73b124a4c2093b04fe8b6b6d9f1 P f08aecdfef264c8b81b70ff01cbed9cd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 998381aa0f734eec9531d0ce425fbe96 in term 1.
I20260806 22:49:53.217837 31003 raft_consensus.cc:2454] T 231ba73b124a4c2093b04fe8b6b6d9f1 P 36075fa9435445349d1c8429845ed592 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 998381aa0f734eec9531d0ce425fbe96 in term 1.
I20260806 22:49:53.217984 31078 leader_election.cc:302] T 231ba73b124a4c2093b04fe8b6b6d9f1 P 998381aa0f734eec9531d0ce425fbe96 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 998381aa0f734eec9531d0ce425fbe96, f08aecdfef264c8b81b70ff01cbed9cd; no voters:
I20260806 22:49:53.218106 31229 raft_consensus.cc:2781] T 231ba73b124a4c2093b04fe8b6b6d9f1 P 998381aa0f734eec9531d0ce425fbe96 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:49:53.218186 31229 raft_consensus.cc:686] T 231ba73b124a4c2093b04fe8b6b6d9f1 P 998381aa0f734eec9531d0ce425fbe96 [term 1 LEADER]: Becoming Leader. State: Replica: 998381aa0f734eec9531d0ce425fbe96, State: Running, Role: LEADER
I20260806 22:49:53.218242 31229 consensus_queue.cc:237] T 231ba73b124a4c2093b04fe8b6b6d9f1 P 998381aa0f734eec9531d0ce425fbe96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f08aecdfef264c8b81b70ff01cbed9cd" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 45763 } } peers { permanent_uuid: "36075fa9435445349d1c8429845ed592" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 34297 } } peers { permanent_uuid: "998381aa0f734eec9531d0ce425fbe96" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 45779 } }
I20260806 22:49:53.218607 31226 tablet_replica.cc:439] T 231ba73b124a4c2093b04fe8b6b6d9f1 P 998381aa0f734eec9531d0ce425fbe96: TxnStatusTablet state changed. Reason: New leader 998381aa0f734eec9531d0ce425fbe96. Latest consensus state: current_term: 1 leader_uuid: "998381aa0f734eec9531d0ce425fbe96" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f08aecdfef264c8b81b70ff01cbed9cd" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 45763 } } peers { permanent_uuid: "36075fa9435445349d1c8429845ed592" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 34297 } } peers { permanent_uuid: "998381aa0f734eec9531d0ce425fbe96" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 45779 } } }
I20260806 22:49:53.218711 31226 tablet_replica.cc:442] T 231ba73b124a4c2093b04fe8b6b6d9f1 P 998381aa0f734eec9531d0ce425fbe96: This TxnStatusTablet replica's current role is: LEADER
I20260806 22:49:53.218856 31265 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 22:49:53.218910 30693 catalog_manager.cc:5495] T 231ba73b124a4c2093b04fe8b6b6d9f1 P 998381aa0f734eec9531d0ce425fbe96 reported cstate change: term changed from 0 to 1, leader changed from <none> to 998381aa0f734eec9531d0ce425fbe96 (127.28.218.195). New cstate: current_term: 1 leader_uuid: "998381aa0f734eec9531d0ce425fbe96" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f08aecdfef264c8b81b70ff01cbed9cd" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 45763 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "36075fa9435445349d1c8429845ed592" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 34297 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "998381aa0f734eec9531d0ce425fbe96" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 45779 } health_report { overall_health: HEALTHY } } }
I20260806 22:49:53.218986 31265 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 22:49:53.225942 31217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:53.223191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55404 (local address 127.28.218.195:45779)
0806 22:49:53.223232 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:53.223234 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:53.223262 (+ 28us) server_negotiation.cc:407] Connection header received
0806 22:49:53.223305 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:53.223306 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:53.223316 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:53.223386 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:49:53.223550 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:53.224311 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:53.225165 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:53.225457 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:53.225537 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:53.225719 (+ 182us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:49:53.225752 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:49:53.225806 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:53.225856 (+ 50us) server_negotiation.cc:299] Negotiation successful
0806 22:49:53.225875 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 22:49:53.225958 31258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:53.223116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.195:45779 (local address 127.0.0.1:55404)
0806 22:49:53.223213 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:53.223222 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:53.223252 (+ 30us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:53.223436 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:53.223437 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:53.223438 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 22:49:53.223482 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:53.223484 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:53.224397 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:53.224398 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:53.225085 (+ 687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:53.225087 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:53.225557 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:53.225558 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:53.225618 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:53.225640 (+ 22us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 22:49:53.225812 (+ 172us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 22:49:53.225818 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 22:49:53.225853 (+ 35us) client_negotiation.cc:240] Negotiation successful
0806 22:49:53.225892 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":68}
I20260806 22:49:53.226838 31003 raft_consensus.cc:1261] T 231ba73b124a4c2093b04fe8b6b6d9f1 P 36075fa9435445349d1c8429845ed592 [term 1 FOLLOWER]: Refusing update from remote peer 998381aa0f734eec9531d0ce425fbe96: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:49:53.226857 30849 raft_consensus.cc:1261] T 231ba73b124a4c2093b04fe8b6b6d9f1 P f08aecdfef264c8b81b70ff01cbed9cd [term 1 FOLLOWER]: Refusing update from remote peer 998381aa0f734eec9531d0ce425fbe96: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:49:53.227059 31229 consensus_queue.cc:1035] T 231ba73b124a4c2093b04fe8b6b6d9f1 P 998381aa0f734eec9531d0ce425fbe96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "36075fa9435445349d1c8429845ed592" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 34297 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:49:53.227221 31226 consensus_queue.cc:1035] T 231ba73b124a4c2093b04fe8b6b6d9f1 P 998381aa0f734eec9531d0ce425fbe96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f08aecdfef264c8b81b70ff01cbed9cd" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 45763 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:49:53.227602 31227 tablet_replica.cc:439] T 231ba73b124a4c2093b04fe8b6b6d9f1 P 36075fa9435445349d1c8429845ed592: TxnStatusTablet state changed. Reason: New leader 998381aa0f734eec9531d0ce425fbe96. Latest consensus state: current_term: 1 leader_uuid: "998381aa0f734eec9531d0ce425fbe96" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f08aecdfef264c8b81b70ff01cbed9cd" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 45763 } } peers { permanent_uuid: "36075fa9435445349d1c8429845ed592" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 34297 } } peers { permanent_uuid: "998381aa0f734eec9531d0ce425fbe96" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 45779 } } }
I20260806 22:49:53.227679 31227 tablet_replica.cc:442] T 231ba73b124a4c2093b04fe8b6b6d9f1 P 36075fa9435445349d1c8429845ed592: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:49:53.228219 31225 tablet_replica.cc:439] T 231ba73b124a4c2093b04fe8b6b6d9f1 P f08aecdfef264c8b81b70ff01cbed9cd: TxnStatusTablet state changed. Reason: New leader 998381aa0f734eec9531d0ce425fbe96. Latest consensus state: current_term: 1 leader_uuid: "998381aa0f734eec9531d0ce425fbe96" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f08aecdfef264c8b81b70ff01cbed9cd" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 45763 } } peers { permanent_uuid: "36075fa9435445349d1c8429845ed592" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 34297 } } peers { permanent_uuid: "998381aa0f734eec9531d0ce425fbe96" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 45779 } } }
I20260806 22:49:53.228307 31225 tablet_replica.cc:442] T 231ba73b124a4c2093b04fe8b6b6d9f1 P f08aecdfef264c8b81b70ff01cbed9cd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:49:53.228483 31229 tablet_replica.cc:439] T 231ba73b124a4c2093b04fe8b6b6d9f1 P 998381aa0f734eec9531d0ce425fbe96: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "998381aa0f734eec9531d0ce425fbe96" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f08aecdfef264c8b81b70ff01cbed9cd" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 45763 } } peers { permanent_uuid: "36075fa9435445349d1c8429845ed592" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 34297 } } peers { permanent_uuid: "998381aa0f734eec9531d0ce425fbe96" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 45779 } } }
I20260806 22:49:53.228569 31229 tablet_replica.cc:442] T 231ba73b124a4c2093b04fe8b6b6d9f1 P 998381aa0f734eec9531d0ce425fbe96: This TxnStatusTablet replica's current role is: LEADER
I20260806 22:49:53.229063 31225 tablet_replica.cc:439] T 231ba73b124a4c2093b04fe8b6b6d9f1 P f08aecdfef264c8b81b70ff01cbed9cd: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "998381aa0f734eec9531d0ce425fbe96" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f08aecdfef264c8b81b70ff01cbed9cd" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 45763 } } peers { permanent_uuid: "36075fa9435445349d1c8429845ed592" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 34297 } } peers { permanent_uuid: "998381aa0f734eec9531d0ce425fbe96" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 45779 } } }
I20260806 22:49:53.229146 31225 tablet_replica.cc:442] T 231ba73b124a4c2093b04fe8b6b6d9f1 P f08aecdfef264c8b81b70ff01cbed9cd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:49:53.230098 31226 tablet_replica.cc:439] T 231ba73b124a4c2093b04fe8b6b6d9f1 P 998381aa0f734eec9531d0ce425fbe96: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "998381aa0f734eec9531d0ce425fbe96" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f08aecdfef264c8b81b70ff01cbed9cd" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 45763 } } peers { permanent_uuid: "36075fa9435445349d1c8429845ed592" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 34297 } } peers { permanent_uuid: "998381aa0f734eec9531d0ce425fbe96" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 45779 } } }
I20260806 22:49:53.230181 31226 tablet_replica.cc:442] T 231ba73b124a4c2093b04fe8b6b6d9f1 P 998381aa0f734eec9531d0ce425fbe96: This TxnStatusTablet replica's current role is: LEADER
I20260806 22:49:53.230690 31227 tablet_replica.cc:439] T 231ba73b124a4c2093b04fe8b6b6d9f1 P 36075fa9435445349d1c8429845ed592: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "998381aa0f734eec9531d0ce425fbe96" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f08aecdfef264c8b81b70ff01cbed9cd" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 45763 } } peers { permanent_uuid: "36075fa9435445349d1c8429845ed592" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 34297 } } peers { permanent_uuid: "998381aa0f734eec9531d0ce425fbe96" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 45779 } } }
I20260806 22:49:53.230778 31227 tablet_replica.cc:442] T 231ba73b124a4c2093b04fe8b6b6d9f1 P 36075fa9435445349d1c8429845ed592: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:49:53.233446 31215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:53.230902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47760 (local address 127.28.218.193:45763)
0806 22:49:53.230942 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:53.230944 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:53.230952 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:49:53.230988 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:53.230989 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:53.231009 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:53.231077 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:49:53.231287 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:53.232000 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:53.232679 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:53.232983 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:53.233053 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:53.233161 (+ 108us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:49:53.233279 (+ 118us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:49:53.233350 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:53.233367 (+ 17us) server_negotiation.cc:299] Negotiation successful
0806 22:49:53.233384 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 22:49:53.238384 31217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:53.235764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.193:52957 (local address 127.28.218.195:45779)
0806 22:49:53.235806 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:53.235808 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:53.235946 (+ 138us) server_negotiation.cc:407] Connection header received
0806 22:49:53.236063 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:53.236063 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:53.236072 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:53.236116 (+ 44us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:49:53.236249 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:53.236707 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:53.237777 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:53.237959 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:53.238010 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:53.238190 (+ 180us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:49:53.238212 (+ 22us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:49:53.238249 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:53.238328 (+ 79us) server_negotiation.cc:299] Negotiation successful
0806 22:49:53.238343 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 22:49:53.238521 31272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:53.235726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.195:45779 (local address 127.28.218.193:52957)
0806 22:49:53.235886 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:53.235896 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:53.235972 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:53.236121 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:53.236121 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:53.236122 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 22:49:53.236187 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:53.236188 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:53.236777 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:53.236778 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:53.237698 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:53.237700 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:53.238032 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:53.238033 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:53.238098 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:53.238112 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 22:49:53.238270 (+ 158us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 22:49:53.238272 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 22:49:53.238326 (+ 54us) client_negotiation.cc:240] Negotiation successful
0806 22:49:53.238364 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":89,"threads_started":1}
I20260806 22:49:53.240556 31003 raft_consensus.cc:1261] T dd6ee574fe5948219ec42c723dd5d263 P 36075fa9435445349d1c8429845ed592 [term 1 FOLLOWER]: Refusing update from remote peer f08aecdfef264c8b81b70ff01cbed9cd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:49:53.240592 31147 raft_consensus.cc:1261] T dd6ee574fe5948219ec42c723dd5d263 P 998381aa0f734eec9531d0ce425fbe96 [term 1 FOLLOWER]: Refusing update from remote peer f08aecdfef264c8b81b70ff01cbed9cd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:49:53.240793 31225 consensus_queue.cc:1035] T dd6ee574fe5948219ec42c723dd5d263 P f08aecdfef264c8b81b70ff01cbed9cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "36075fa9435445349d1c8429845ed592" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 34297 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:49:53.240978 31251 consensus_queue.cc:1035] T dd6ee574fe5948219ec42c723dd5d263 P f08aecdfef264c8b81b70ff01cbed9cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "998381aa0f734eec9531d0ce425fbe96" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 45779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:49:53.242048 31233 mvcc.cc:205] Tried to move back new op lower bound from 7315687805911662592 to 7315687805397692416. Current Snapshot: MvccSnapshot[applied={T|T < 7315687805911662592}]
I20260806 22:49:53.249228 31215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:53.246791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.195:33969 (local address 127.28.218.193:45763)
0806 22:49:53.246843 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:53.246845 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:53.246853 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:49:53.246915 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:53.246916 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:53.246925 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:53.246969 (+ 44us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:49:53.247062 (+ 93us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:53.247537 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:53.248490 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:53.248784 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:53.248858 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:53.249015 (+ 157us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:49:53.249047 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:49:53.249101 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:53.249145 (+ 44us) server_negotiation.cc:299] Negotiation successful
0806 22:49:53.249164 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 22:49:53.249238 31278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:53.246629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.193:45763 (local address 127.28.218.195:33969)
0806 22:49:53.246782 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:53.246792 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:53.246850 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:53.246970 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:53.246971 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:53.246972 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 22:49:53.247012 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:53.247013 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:53.247631 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:53.247632 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:53.248394 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:53.248396 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:53.248877 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:53.248878 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:53.248928 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:53.248939 (+ 11us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 22:49:53.249107 (+ 168us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 22:49:53.249108 (+ 1us) client_negotiation.cc:769] Sending connection context
0806 22:49:53.249142 (+ 34us) client_negotiation.cc:240] Negotiation successful
0806 22:49:53.249174 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":72,"threads_started":1}
I20260806 22:49:53.256353 30694 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45642:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:49:53.256453 30694 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45642:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:49:53.257072 30694 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5e2fa0b04c9a4d41a3a5532eb98e4777: Sending DeleteTablet for 3 replicas of tablet dd6ee574fe5948219ec42c723dd5d263
I20260806 22:49:53.257292 30837 tablet_service.cc:1514] Processing DeleteTablet for tablet dd6ee574fe5948219ec42c723dd5d263 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:49:53 UTC) from {username='kudu', principal='kudu/127.28.218.254@KRBTEST.COM'} at 127.0.0.1:47754
I20260806 22:49:53.257378 31242 tablet_replica.cc:330] T dd6ee574fe5948219ec42c723dd5d263 P f08aecdfef264c8b81b70ff01cbed9cd: stopping tablet replica
I20260806 22:49:53.257423 31242 raft_consensus.cc:2229] T dd6ee574fe5948219ec42c723dd5d263 P f08aecdfef264c8b81b70ff01cbed9cd [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:49:53.257421 31127 tablet_service.cc:1514] Processing DeleteTablet for tablet dd6ee574fe5948219ec42c723dd5d263 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:49:53 UTC) from {username='kudu', principal='kudu/127.28.218.254@KRBTEST.COM'} at 127.0.0.1:55384
I20260806 22:49:53.257494 31242 raft_consensus.cc:2258] T dd6ee574fe5948219ec42c723dd5d263 P f08aecdfef264c8b81b70ff01cbed9cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:49:53.257508 31244 tablet_replica.cc:330] T dd6ee574fe5948219ec42c723dd5d263 P 998381aa0f734eec9531d0ce425fbe96: stopping tablet replica
I20260806 22:49:53.257552 31244 raft_consensus.cc:2229] T dd6ee574fe5948219ec42c723dd5d263 P 998381aa0f734eec9531d0ce425fbe96 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:49:53.257581 31244 raft_consensus.cc:2258] T dd6ee574fe5948219ec42c723dd5d263 P 998381aa0f734eec9531d0ce425fbe96 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:49:53.257570 30981 tablet_service.cc:1514] Processing DeleteTablet for tablet dd6ee574fe5948219ec42c723dd5d263 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:49:53 UTC) from {username='kudu', principal='kudu/127.28.218.254@KRBTEST.COM'} at 127.0.0.1:52420
I20260806 22:49:53.257704 31243 tablet_replica.cc:330] T dd6ee574fe5948219ec42c723dd5d263 P 36075fa9435445349d1c8429845ed592: stopping tablet replica
I20260806 22:49:53.257745 31243 raft_consensus.cc:2229] T dd6ee574fe5948219ec42c723dd5d263 P 36075fa9435445349d1c8429845ed592 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:49:53.257790 31243 raft_consensus.cc:2258] T dd6ee574fe5948219ec42c723dd5d263 P 36075fa9435445349d1c8429845ed592 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:49:53.257869 31244 ts_tablet_manager.cc:1905] T dd6ee574fe5948219ec42c723dd5d263 P 998381aa0f734eec9531d0ce425fbe96: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:49:53.258287 31243 ts_tablet_manager.cc:1905] T dd6ee574fe5948219ec42c723dd5d263 P 36075fa9435445349d1c8429845ed592: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:49:53.259666 31244 ts_tablet_manager.cc:1918] T dd6ee574fe5948219ec42c723dd5d263 P 998381aa0f734eec9531d0ce425fbe96: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 22:49:53.259718 31244 log.cc:1196] T dd6ee574fe5948219ec42c723dd5d263 P 998381aa0f734eec9531d0ce425fbe96: Deleting WAL directory at /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056586788339-29547-0/minicluster-data/ts-2/wal/wals/dd6ee574fe5948219ec42c723dd5d263
I20260806 22:49:53.259876 31243 ts_tablet_manager.cc:1918] T dd6ee574fe5948219ec42c723dd5d263 P 36075fa9435445349d1c8429845ed592: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 22:49:53.259936 31243 log.cc:1196] T dd6ee574fe5948219ec42c723dd5d263 P 36075fa9435445349d1c8429845ed592: Deleting WAL directory at /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056586788339-29547-0/minicluster-data/ts-1/wal/wals/dd6ee574fe5948219ec42c723dd5d263
I20260806 22:49:53.259943 31244 ts_tablet_manager.cc:1939] T dd6ee574fe5948219ec42c723dd5d263 P 998381aa0f734eec9531d0ce425fbe96: Deleting consensus metadata
I20260806 22:49:53.260159 31243 ts_tablet_manager.cc:1939] T dd6ee574fe5948219ec42c723dd5d263 P 36075fa9435445349d1c8429845ed592: Deleting consensus metadata
I20260806 22:49:53.260228 30678 catalog_manager.cc:4841] TS 998381aa0f734eec9531d0ce425fbe96 (127.28.218.195:45779): tablet dd6ee574fe5948219ec42c723dd5d263 (table test-table [id=8abe36678bf247f7b0cc42691b0f5956]) successfully deleted
I20260806 22:49:53.260422 30678 catalog_manager.cc:4841] TS 36075fa9435445349d1c8429845ed592 (127.28.218.194:34297): tablet dd6ee574fe5948219ec42c723dd5d263 (table test-table [id=8abe36678bf247f7b0cc42691b0f5956]) successfully deleted
I20260806 22:49:53.261271 31242 ts_tablet_manager.cc:1905] T dd6ee574fe5948219ec42c723dd5d263 P f08aecdfef264c8b81b70ff01cbed9cd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:49:53.262779 31242 ts_tablet_manager.cc:1918] T dd6ee574fe5948219ec42c723dd5d263 P f08aecdfef264c8b81b70ff01cbed9cd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 22:49:53.262828 31242 log.cc:1196] T dd6ee574fe5948219ec42c723dd5d263 P f08aecdfef264c8b81b70ff01cbed9cd: Deleting WAL directory at /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056586788339-29547-0/minicluster-data/ts-0/wal/wals/dd6ee574fe5948219ec42c723dd5d263
I20260806 22:49:53.263047 31242 ts_tablet_manager.cc:1939] T dd6ee574fe5948219ec42c723dd5d263 P f08aecdfef264c8b81b70ff01cbed9cd: Deleting consensus metadata
I20260806 22:49:53.263366 30679 catalog_manager.cc:4841] TS f08aecdfef264c8b81b70ff01cbed9cd (127.28.218.193:45763): tablet dd6ee574fe5948219ec42c723dd5d263 (table test-table [id=8abe36678bf247f7b0cc42691b0f5956]) successfully deleted
Aug 06 22:49:53 dist-test-slave-1xrh krb5kdc[30645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056592, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.28.218.193@KRBTEST.COM
I20260806 22:49:53.272776 31215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:53.261117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47774 (local address 127.28.218.193:45763)
0806 22:49:53.261156 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:53.261158 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:53.261382 (+ 224us) server_negotiation.cc:407] Connection header received
0806 22:49:53.261491 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:53.261492 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:53.261501 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:53.261530 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:53.262297 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:53.263032 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:53.264150 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:53.264438 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:53.264516 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:53.269294 (+ 4778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:53.269320 (+ 26us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:53.269320 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:49:53.269327 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:53.271446 (+ 2119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:53.271859 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:53.271862 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:53.271862 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:53.271944 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:53.272154 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:53.272155 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:53.272155 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:53.272321 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:53.272472 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:53.272588 (+ 116us) server_negotiation.cc:299] Negotiation successful
0806 22:49:53.272670 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
W20260806 22:49:53.273089 30893 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47774
I20260806 22:49:53.282352 31259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:53.273957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45646 (local address 127.28.218.254:43095)
0806 22:49:53.274011 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:53.274013 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:53.275270 (+ 1257us) server_negotiation.cc:407] Connection header received
0806 22:49:53.275371 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:53.275372 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:53.275384 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:53.275441 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:53.276072 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:53.276836 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:53.277835 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:53.278152 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:53.278222 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:53.279593 (+ 1371us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:53.279594 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:53.279595 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:49:53.279602 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:53.281207 (+ 1605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:53.281586 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:53.281587 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:53.281587 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:53.281646 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:53.281848 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:53.281849 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:53.281849 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:53.281978 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:53.282075 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:53.282181 (+ 106us) server_negotiation.cc:299] Negotiation successful
0806 22:49:53.282263 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
W20260806 22:49:53.282835 30694 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45646
I20260806 22:49:53.283565 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 30770
I20260806 22:49:53.290352 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 30915
I20260806 22:49:53.296208 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 31060
I20260806 22:49:53.303085 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 30659
2026-08-06T22:49:53Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2540 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056586788339-29547-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:49:53 dist-test-slave-1xrh krb5kdc[31289](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:49:53 dist-test-slave-1xrh krb5kdc[31289](info): set up 2 sockets
Aug 06 22:49:53 dist-test-slave-1xrh krb5kdc[31289](info): commencing operation
krb5kdc: starting...
W20260806 22:49:54.346524 29547 mini_kdc.cc:121] Time spent starting KDC: real 1.022s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056586788339-29547-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056586788339-29547-0/krb5kdc/test-user.keytab.
Aug 06 22:49:54 dist-test-slave-1xrh krb5kdc[31289](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056594, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:49:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:49:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.218.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.218.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.218.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab.
Entry for principal kudu/127.28.218.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab.
I20260806 22:49:54.466238 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056586788339-29547-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056586788339-29547-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056586788339-29547-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.218.254:38323
--webserver_interface=127.28.218.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab
--principal=kudu/127.28.218.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.218.212:41221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.218.254:38323
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:49:54.558373 31302 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:49:54.558679 31302 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:49:54.558723 31302 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:49:54.561000 31302 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:49:54.561062 31302 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 22:49:54.561088 31302 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:49:54.561112 31302 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:49:54.561144 31302 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:49:54.561175 31302 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:49:54.563779 31302 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:41221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056586788339-29547-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056586788339-29547-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.218.254:38323
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab
--principal=kudu/127.28.218.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.218.254:38323
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.218.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31302
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056586788339-29547-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:49:54.564110 31302 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:49:54.564383 31302 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:49:54.569696 31308 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:54.569747 31310 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:54.569852 31307 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:49:54.569911 31302 server_base.cc:1034] running on GCE node
I20260806 22:49:54.570215 31302 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:49:54.570573 31302 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:49:54.571750 31302 hybrid_clock.cc:648] HybridClock initialized: now 1786056594571743 us; error 30 us; skew 500 ppm
Aug 06 22:49:54 dist-test-slave-1xrh krb5kdc[31289](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056594, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:49:54.574326 31302 init.cc:377] Logged in from keytab as kudu/127.28.218.254@KRBTEST.COM (short username kudu)
I20260806 22:49:54.574710 31302 webserver.cc:460] Webserver started at http://127.28.218.254:41035/ using document root <none> and password file <none>
I20260806 22:49:54.574882 31302 fs_manager.cc:359] Metadata directory not provided
I20260806 22:49:54.574927 31302 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:49:54.575012 31302 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:49:54.576099 31302 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056586788339-29547-0/minicluster-data/master-0/data/instance:
uuid: "d5a2faa728f142b79312f4cb47b13e55"
format_stamp: "Formatted at 2026-08-06 22:49:54 on dist-test-slave-1xrh"
server_key: "96d8cec601f4ab2b4899bae154c6b281"
server_key_iv: "309087be2fdcc5324cf1b65ad9ba93d4"
server_key_version: "encryptionkey@0"
I20260806 22:49:54.576428 31302 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056586788339-29547-0/minicluster-data/master-0/wal/instance:
uuid: "d5a2faa728f142b79312f4cb47b13e55"
format_stamp: "Formatted at 2026-08-06 22:49:54 on dist-test-slave-1xrh"
server_key: "96d8cec601f4ab2b4899bae154c6b281"
server_key_iv: "309087be2fdcc5324cf1b65ad9ba93d4"
server_key_version: "encryptionkey@0"
I20260806 22:49:54.577663 31302 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:49:54.578292 31317 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:54.578454 31302 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:49:54.578536 31302 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056586788339-29547-0/minicluster-data/master-0/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056586788339-29547-0/minicluster-data/master-0/wal
uuid: "d5a2faa728f142b79312f4cb47b13e55"
format_stamp: "Formatted at 2026-08-06 22:49:54 on dist-test-slave-1xrh"
server_key: "96d8cec601f4ab2b4899bae154c6b281"
server_key_iv: "309087be2fdcc5324cf1b65ad9ba93d4"
server_key_version: "encryptionkey@0"
I20260806 22:49:54.578588 31302 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056586788339-29547-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056586788339-29547-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056586788339-29547-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:49:54.595394 31302 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:49:54.597157 31302 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:49:54.597266 31302 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:49:54.601742 31302 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.254:38323
I20260806 22:49:54.601832 31383 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.254:38323 every 8 connection(s)
I20260806 22:49:54.602142 31302 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
I20260806 22:49:54.602807 31384 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:49:54.605152 31384 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d5a2faa728f142b79312f4cb47b13e55: Bootstrap starting.
I20260806 22:49:54.605756 31384 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d5a2faa728f142b79312f4cb47b13e55: Neither blocks nor log segments found. Creating new log.
I20260806 22:49:54.606050 31384 log.cc:824] T 00000000000000000000000000000000 P d5a2faa728f142b79312f4cb47b13e55: Log is configured to *not* fsync() on all Append() calls
I20260806 22:49:54.606725 31384 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d5a2faa728f142b79312f4cb47b13e55: No bootstrap required, opened a new log
I20260806 22:49:54.608863 31384 raft_consensus.cc:348] T 00000000000000000000000000000000 P d5a2faa728f142b79312f4cb47b13e55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5a2faa728f142b79312f4cb47b13e55" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 38323 } }
I20260806 22:49:54.609001 31384 raft_consensus.cc:374] T 00000000000000000000000000000000 P d5a2faa728f142b79312f4cb47b13e55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:49:54.609047 31384 raft_consensus.cc:729] T 00000000000000000000000000000000 P d5a2faa728f142b79312f4cb47b13e55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5a2faa728f142b79312f4cb47b13e55, State: Initialized, Role: FOLLOWER
I20260806 22:49:54.609200 31384 consensus_queue.cc:260] T 00000000000000000000000000000000 P d5a2faa728f142b79312f4cb47b13e55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5a2faa728f142b79312f4cb47b13e55" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 38323 } }
I20260806 22:49:54.609264 31384 raft_consensus.cc:388] T 00000000000000000000000000000000 P d5a2faa728f142b79312f4cb47b13e55 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:49:54.609306 31384 raft_consensus.cc:482] T 00000000000000000000000000000000 P d5a2faa728f142b79312f4cb47b13e55 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:49:54.609355 31384 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d5a2faa728f142b79312f4cb47b13e55 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:49:54.609869 31384 raft_consensus.cc:504] T 00000000000000000000000000000000 P d5a2faa728f142b79312f4cb47b13e55 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5a2faa728f142b79312f4cb47b13e55" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 38323 } }
I20260806 22:49:54.609982 31384 leader_election.cc:302] T 00000000000000000000000000000000 P d5a2faa728f142b79312f4cb47b13e55 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d5a2faa728f142b79312f4cb47b13e55; no voters:
I20260806 22:49:54.610162 31384 leader_election.cc:288] T 00000000000000000000000000000000 P d5a2faa728f142b79312f4cb47b13e55 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:49:54.610198 31387 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d5a2faa728f142b79312f4cb47b13e55 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:49:54.610348 31387 raft_consensus.cc:686] T 00000000000000000000000000000000 P d5a2faa728f142b79312f4cb47b13e55 [term 1 LEADER]: Becoming Leader. State: Replica: d5a2faa728f142b79312f4cb47b13e55, State: Running, Role: LEADER
I20260806 22:49:54.610458 31384 sys_catalog.cc:564] T 00000000000000000000000000000000 P d5a2faa728f142b79312f4cb47b13e55 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:49:54.610493 31387 consensus_queue.cc:237] T 00000000000000000000000000000000 P d5a2faa728f142b79312f4cb47b13e55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5a2faa728f142b79312f4cb47b13e55" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 38323 } }
I20260806 22:49:54.610879 31389 sys_catalog.cc:455] T 00000000000000000000000000000000 P d5a2faa728f142b79312f4cb47b13e55 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d5a2faa728f142b79312f4cb47b13e55. Latest consensus state: current_term: 1 leader_uuid: "d5a2faa728f142b79312f4cb47b13e55" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5a2faa728f142b79312f4cb47b13e55" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 38323 } } }
I20260806 22:49:54.610874 31388 sys_catalog.cc:455] T 00000000000000000000000000000000 P d5a2faa728f142b79312f4cb47b13e55 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d5a2faa728f142b79312f4cb47b13e55" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5a2faa728f142b79312f4cb47b13e55" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 38323 } } }
I20260806 22:49:54.611055 31388 sys_catalog.cc:458] T 00000000000000000000000000000000 P d5a2faa728f142b79312f4cb47b13e55 [sys.catalog]: This master's current role is: LEADER
I20260806 22:49:54.611092 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 31302
I20260806 22:49:54.611202 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056586788339-29547-0/minicluster-data/master-0/wal/instance
I20260806 22:49:54.611047 31389 sys_catalog.cc:458] T 00000000000000000000000000000000 P d5a2faa728f142b79312f4cb47b13e55 [sys.catalog]: This master's current role is: LEADER
I20260806 22:49:54.611284 31392 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:49:54.611387 29547 external_mini_cluster.cc:1428] Setting key bcf2e4ec2bde810162b390cb7eec98ab
I20260806 22:49:54.611793 31392 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:49:54.613358 31392 catalog_manager.cc:1347] Generated new cluster ID: 7ab314cb3aea4a57bf700559fd34d690
I20260806 22:49:54.613407 31392 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:49:54.618135 31392 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:49:54.618892 31392 catalog_manager.cc:1504] Loading token signing keys...
Aug 06 22:49:54 dist-test-slave-1xrh krb5kdc[31289](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056594, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.218.254@KRBTEST.COM
I20260806 22:49:54.623853 31400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:54.612076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32902 (local address 127.28.218.254:38323)
0806 22:49:54.612267 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:54.612276 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:54.612690 (+ 414us) server_negotiation.cc:407] Connection header received
0806 22:49:54.612982 (+ 292us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:54.612990 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:54.613062 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:54.613991 (+ 929us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:54.614109 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:54.615503 (+ 1394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:54.616584 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:54.617229 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:54.617341 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:54.620191 (+ 2850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:54.620195 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:54.620195 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:49:54.620206 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:54.622097 (+ 1891us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:54.622502 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:54.622505 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:54.622505 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:54.622613 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:54.622827 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:54.622828 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:54.622828 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:54.623073 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:54.623265 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:54.623375 (+ 110us) server_negotiation.cc:299] Negotiation successful
0806 22:49:54.623618 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":92,"threads_started":1}
I20260806 22:49:54.634588 31392 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d5a2faa728f142b79312f4cb47b13e55: Generated new TSK 0
I20260806 22:49:54.634752 31392 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.28.218.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.218.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.218.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.193.keytab.
Entry for principal kudu/127.28.218.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.193.keytab.
I20260806 22:49:54.662147 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056586788339-29547-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056586788339-29547-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056586788339-29547-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056586788339-29547-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.218.193:0
--local_ip_for_outbound_sockets=127.28.218.193
--webserver_interface=127.28.218.193
--webserver_port=0
--tserver_master_addrs=127.28.218.254:38323
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.193.keytab
--principal=kudu/127.28.218.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.218.212:41221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:49:54.741961 31410 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:49:54.742154 31410 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:49:54.742178 31410 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:49:54.743315 31410 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:49:54.743355 31410 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:49:54.743376 31410 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 22:49:54.743428 31410 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 22:49:54.743453 31410 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.218.193
I20260806 22:49:54.744895 31410 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:41221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056586788339-29547-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056586788339-29547-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.193.keytab
--principal=kudu/127.28.218.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.218.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056586788339-29547-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.218.193
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.28.218.254:38323
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31410
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.218.193
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056586788339-29547-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:49:54.745126 31410 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:49:54.745344 31410 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:49:54.750140 31415 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:54.750152 31416 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:54.750147 31419 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:49:54.750478 31410 server_base.cc:1034] running on GCE node
I20260806 22:49:54.750674 31410 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:49:54.750917 31410 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:49:54.752074 31410 hybrid_clock.cc:648] HybridClock initialized: now 1786056594752052 us; error 58 us; skew 500 ppm
Aug 06 22:49:54 dist-test-slave-1xrh krb5kdc[31289](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056594, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:49:54.754410 31410 init.cc:377] Logged in from keytab as kudu/127.28.218.193@KRBTEST.COM (short username kudu)
I20260806 22:49:54.754927 31410 webserver.cc:460] Webserver started at http://127.28.218.193:41539/ using document root <none> and password file <none>
I20260806 22:49:54.755216 31410 fs_manager.cc:359] Metadata directory not provided
I20260806 22:49:54.755285 31410 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:49:54.755421 31410 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:49:54.756433 31410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056586788339-29547-0/minicluster-data/ts-0/data/instance:
uuid: "9edf436137414f54b26af5a8ec47d81c"
format_stamp: "Formatted at 2026-08-06 22:49:54 on dist-test-slave-1xrh"
server_key: "be0f402cf5b5aad058dc6b980e1bc311"
server_key_iv: "7f1a6807812ec06e5920b0be8ff8d20b"
server_key_version: "encryptionkey@0"
I20260806 22:49:54.756793 31410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056586788339-29547-0/minicluster-data/ts-0/wal/instance:
uuid: "9edf436137414f54b26af5a8ec47d81c"
format_stamp: "Formatted at 2026-08-06 22:49:54 on dist-test-slave-1xrh"
server_key: "be0f402cf5b5aad058dc6b980e1bc311"
server_key_iv: "7f1a6807812ec06e5920b0be8ff8d20b"
server_key_version: "encryptionkey@0"
I20260806 22:49:54.758131 31410 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:49:54.758821 31426 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:54.758986 31410 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:49:54.759078 31410 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056586788339-29547-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056586788339-29547-0/minicluster-data/ts-0/wal
uuid: "9edf436137414f54b26af5a8ec47d81c"
format_stamp: "Formatted at 2026-08-06 22:49:54 on dist-test-slave-1xrh"
server_key: "be0f402cf5b5aad058dc6b980e1bc311"
server_key_iv: "7f1a6807812ec06e5920b0be8ff8d20b"
server_key_version: "encryptionkey@0"
I20260806 22:49:54.759166 31410 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056586788339-29547-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056586788339-29547-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056586788339-29547-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:49:54.768989 31410 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:49:54.770713 31410 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:49:54.770867 31410 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:49:54.771445 31410 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:49:54.771505 31410 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:54.771543 31410 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:49:54.771569 31410 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 22:49:54 dist-test-slave-1xrh krb5kdc[31289](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056594, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.193@KRBTEST.COM for kudu/127.28.218.254@KRBTEST.COM
I20260806 22:49:54.778824 31410 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.193:35231
I20260806 22:49:54.778916 31549 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.193:35231 every 8 connection(s)
I20260806 22:49:54.779223 31410 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056586788339-29547-0/minicluster-data/ts-0/data/info.pb
I20260806 22:49:54.781243 31400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:54.771724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.193:51093 (local address 127.28.218.254:38323)
0806 22:49:54.771775 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:54.771778 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:54.772034 (+ 256us) server_negotiation.cc:407] Connection header received
0806 22:49:54.772465 (+ 431us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:54.772467 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:54.772493 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:54.772573 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:54.773098 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:54.773599 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:54.774637 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:54.774862 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:54.774941 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:54.778351 (+ 3410us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:54.778353 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:54.778353 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:49:54.778361 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:54.780040 (+ 1679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:54.780409 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:54.780410 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:54.780410 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:54.780462 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:54.780652 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:54.780652 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:54.780652 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:54.780741 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:54.780786 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:54.781054 (+ 268us) server_negotiation.cc:299] Negotiation successful
0806 22:49:54.781183 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 22:49:54.781358 31437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:54.771724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:38323 (local address 127.28.218.193:51093)
0806 22:49:54.771883 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:54.771937 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:54.772372 (+ 435us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:54.772615 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:54.772618 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:54.772831 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:49:54.772999 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:54.773003 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:54.773724 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:54.773725 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:54.774542 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:54.774544 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:54.774951 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:54.774953 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:54.775088 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:54.775663 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:49:54.775673 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:49:54.778247 (+ 2574us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:49:54.780106 (+ 1859us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:54.780111 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:54.780112 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:54.780342 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:54.780524 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:54.780525 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:54.780525 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:54.780613 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:54.780789 (+ 176us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:49:54.780792 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:49:54.780938 (+ 146us) client_negotiation.cc:769] Sending connection context
0806 22:49:54.781023 (+ 85us) client_negotiation.cc:240] Negotiation successful
0806 22:49:54.781180 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":99,"threads_started":1}
I20260806 22:49:54.781819 31550 heartbeater.cc:347] Connected to a master server at 127.28.218.254:38323
I20260806 22:49:54.781900 31550 heartbeater.cc:464] Registering TS with master...
I20260806 22:49:54.782091 31550 heartbeater.cc:511] Master 127.28.218.254:38323 requested a full tablet report, sending...
I20260806 22:49:54.782606 31333 ts_manager.cc:194] Registered new tserver with Master: 9edf436137414f54b26af5a8ec47d81c (127.28.218.193:35231)
I20260806 22:49:54.783382 31333 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.218.193@KRBTEST.COM'} at 127.28.218.193:51093
I20260806 22:49:54.787060 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 31410
I20260806 22:49:54.787185 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056586788339-29547-0/minicluster-data/ts-0/wal/instance
I20260806 22:49:54.787422 29547 external_mini_cluster.cc:1428] Setting key 94256a06df9f80fa72f641b22431e93b
I20260806 22:49:54.787979 31400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:54.782851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.193:33903 (local address 127.28.218.254:38323)
0806 22:49:54.782883 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:54.782885 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:54.782987 (+ 102us) server_negotiation.cc:407] Connection header received
0806 22:49:54.783029 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:54.783029 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:54.783038 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:54.783099 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:54.783425 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:54.784079 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:54.784707 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:54.784873 (+ 166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:54.784919 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:54.785757 (+ 838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:54.785758 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:54.785758 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:49:54.785763 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:54.786817 (+ 1054us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:54.787231 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:54.787232 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:54.787232 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:54.787291 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:54.787521 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:54.787522 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:54.787522 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:54.787625 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:54.787698 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:54.787855 (+ 157us) server_negotiation.cc:299] Negotiation successful
0806 22:49:54.787922 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260806 22:49:54.788021 31556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:54.782786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:38323 (local address 127.28.218.193:33903)
0806 22:49:54.782919 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:54.782929 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:54.782984 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:54.783113 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:54.783113 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:54.783307 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:49:54.783357 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:54.783358 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:54.784182 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:54.784183 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:54.784642 (+ 459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:54.784643 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:54.784946 (+ 303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:54.784946 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:54.785000 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:54.785422 (+ 422us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:49:54.785427 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:49:54.785689 (+ 262us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:49:54.786898 (+ 1209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:54.786899 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:54.786899 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:54.787156 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:54.787360 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:54.787361 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:54.787361 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:54.787428 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:54.787705 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:49:54.787706 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:49:54.787753 (+ 47us) client_negotiation.cc:769] Sending connection context
0806 22:49:54.787824 (+ 71us) client_negotiation.cc:240] Negotiation successful
0806 22:49:54.787917 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":79,"threads_started":1}
WARNING: no policy specified for kudu/127.28.218.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.218.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.218.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.194.keytab.
Entry for principal kudu/127.28.218.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.194.keytab.
I20260806 22:49:54.813719 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056586788339-29547-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056586788339-29547-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056586788339-29547-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056586788339-29547-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.218.194:0
--local_ip_for_outbound_sockets=127.28.218.194
--webserver_interface=127.28.218.194
--webserver_port=0
--tserver_master_addrs=127.28.218.254:38323
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.194.keytab
--principal=kudu/127.28.218.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.218.212:41221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:49:54.895363 31559 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:49:54.895548 31559 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:49:54.895572 31559 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:49:54.896673 31559 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:49:54.896713 31559 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:49:54.896755 31559 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 22:49:54.896785 31559 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 22:49:54.896814 31559 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.218.194
I20260806 22:49:54.898247 31559 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:41221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056586788339-29547-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056586788339-29547-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.194.keytab
--principal=kudu/127.28.218.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.218.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056586788339-29547-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.218.194
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.28.218.254:38323
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31559
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.218.194
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056586788339-29547-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:49:54.898490 31559 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:49:54.898703 31559 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:49:54.903465 31565 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:54.903522 31564 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:54.903523 31567 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:49:54.903680 31559 server_base.cc:1034] running on GCE node
I20260806 22:49:54.904003 31559 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:49:54.904237 31559 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:49:54.905412 31559 hybrid_clock.cc:648] HybridClock initialized: now 1786056594905395 us; error 33 us; skew 500 ppm
Aug 06 22:49:54 dist-test-slave-1xrh krb5kdc[31289](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056594, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:49:54.908185 31559 init.cc:377] Logged in from keytab as kudu/127.28.218.194@KRBTEST.COM (short username kudu)
I20260806 22:49:54.908560 31559 webserver.cc:460] Webserver started at http://127.28.218.194:46207/ using document root <none> and password file <none>
I20260806 22:49:54.908811 31559 fs_manager.cc:359] Metadata directory not provided
I20260806 22:49:54.908860 31559 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:49:54.909017 31559 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:49:54.910084 31559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056586788339-29547-0/minicluster-data/ts-1/data/instance:
uuid: "8da8827f19c842f6bfae8a9ee27ef69a"
format_stamp: "Formatted at 2026-08-06 22:49:54 on dist-test-slave-1xrh"
server_key: "c30c1b9448243e91574875b712542004"
server_key_iv: "e331d93b313fbe5a6d6ca8d674022c86"
server_key_version: "encryptionkey@0"
I20260806 22:49:54.910425 31559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056586788339-29547-0/minicluster-data/ts-1/wal/instance:
uuid: "8da8827f19c842f6bfae8a9ee27ef69a"
format_stamp: "Formatted at 2026-08-06 22:49:54 on dist-test-slave-1xrh"
server_key: "c30c1b9448243e91574875b712542004"
server_key_iv: "e331d93b313fbe5a6d6ca8d674022c86"
server_key_version: "encryptionkey@0"
I20260806 22:49:54.912019 31559 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:49:54.912786 31574 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:54.912978 31559 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:54.913045 31559 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056586788339-29547-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056586788339-29547-0/minicluster-data/ts-1/wal
uuid: "8da8827f19c842f6bfae8a9ee27ef69a"
format_stamp: "Formatted at 2026-08-06 22:49:54 on dist-test-slave-1xrh"
server_key: "c30c1b9448243e91574875b712542004"
server_key_iv: "e331d93b313fbe5a6d6ca8d674022c86"
server_key_version: "encryptionkey@0"
I20260806 22:49:54.913130 31559 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056586788339-29547-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056586788339-29547-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056586788339-29547-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:49:54.920960 31559 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:49:54.922606 31559 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:49:54.922747 31559 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:49:54.923290 31559 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:49:54.923349 31559 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:54.923393 31559 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:49:54.923420 31559 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 22:49:54 dist-test-slave-1xrh krb5kdc[31289](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056594, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.194@KRBTEST.COM for kudu/127.28.218.254@KRBTEST.COM
I20260806 22:49:54.930235 31559 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.194:33853
I20260806 22:49:54.930339 31696 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.194:33853 every 8 connection(s)
I20260806 22:49:54.930609 31559 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056586788339-29547-0/minicluster-data/ts-1/data/info.pb
I20260806 22:49:54.933938 31400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:54.923693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.194:43219 (local address 127.28.218.254:38323)
0806 22:49:54.923768 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:54.923770 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:54.923972 (+ 202us) server_negotiation.cc:407] Connection header received
0806 22:49:54.924376 (+ 404us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:54.924377 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:54.924391 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:54.924476 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:54.925229 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:54.926074 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:54.927676 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:54.928064 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:54.928152 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:54.930758 (+ 2606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:54.930760 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:54.930760 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:49:54.930768 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:54.932445 (+ 1677us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:54.932990 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:54.932992 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:54.932992 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:54.933052 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:54.933309 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:54.933310 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:54.933310 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:54.933433 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:54.933510 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:54.933766 (+ 256us) server_negotiation.cc:299] Negotiation successful
0806 22:49:54.933848 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 22:49:54.934129 31583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:54.923670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:38323 (local address 127.28.218.194:43219)
0806 22:49:54.923832 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:54.923869 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:54.924287 (+ 418us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:54.924547 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:54.924551 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:54.924846 (+ 295us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:49:54.925124 (+ 278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:54.925130 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:54.926205 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:54.926206 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:54.927579 (+ 1373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:54.927583 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:54.928174 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:54.928175 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:54.928310 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:54.928796 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:49:54.928804 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:49:54.930655 (+ 1851us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:49:54.932537 (+ 1882us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:54.932544 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:54.932547 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:54.932915 (+ 368us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:54.933132 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:54.933133 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:54.933133 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:54.933240 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:54.933500 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:49:54.933507 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:49:54.933640 (+ 133us) client_negotiation.cc:769] Sending connection context
0806 22:49:54.933745 (+ 105us) client_negotiation.cc:240] Negotiation successful
0806 22:49:54.933895 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":88,"threads_started":1}
I20260806 22:49:54.934603 31697 heartbeater.cc:347] Connected to a master server at 127.28.218.254:38323
I20260806 22:49:54.934700 31697 heartbeater.cc:464] Registering TS with master...
I20260806 22:49:54.934857 31697 heartbeater.cc:511] Master 127.28.218.254:38323 requested a full tablet report, sending...
I20260806 22:49:54.935262 31333 ts_manager.cc:194] Registered new tserver with Master: 8da8827f19c842f6bfae8a9ee27ef69a (127.28.218.194:33853)
I20260806 22:49:54.935667 31333 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.218.194@KRBTEST.COM'} at 127.28.218.194:43219
I20260806 22:49:54.938108 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 31559
I20260806 22:49:54.938216 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056586788339-29547-0/minicluster-data/ts-1/wal/instance
I20260806 22:49:54.938403 29547 external_mini_cluster.cc:1428] Setting key e92631be620e14bb7d625f9d387e0a2e
I20260806 22:49:54.941135 31703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:54.935625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:38323 (local address 127.28.218.194:39629)
0806 22:49:54.935793 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:54.935804 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:54.935859 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:54.936021 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:54.936022 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:54.936217 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:49:54.936273 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:54.936274 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:54.936912 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:54.936913 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:54.937435 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:54.937437 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:54.937762 (+ 325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:54.937762 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:54.937827 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:54.938159 (+ 332us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:49:54.938164 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:49:54.938516 (+ 352us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:49:54.940107 (+ 1591us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:54.940108 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:54.940108 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:54.940329 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:54.940511 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:54.940512 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:54.940512 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:54.940580 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:54.940921 (+ 341us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:49:54.940922 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:49:54.940956 (+ 34us) client_negotiation.cc:769] Sending connection context
0806 22:49:54.941013 (+ 57us) client_negotiation.cc:240] Negotiation successful
0806 22:49:54.941079 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":76,"threads_started":1}
I20260806 22:49:54.941222 31400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:54.935711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.194:39629 (local address 127.28.218.254:38323)
0806 22:49:54.935751 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:54.935752 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:54.935864 (+ 112us) server_negotiation.cc:407] Connection header received
0806 22:49:54.935916 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:54.935917 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:54.935927 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:54.936002 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:54.936354 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:54.936833 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:54.937528 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:54.937701 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:54.937751 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:54.938618 (+ 867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:54.938619 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:54.938620 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:49:54.938627 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:54.940023 (+ 1396us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:54.940406 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:54.940407 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:54.940407 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:54.940452 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:54.940654 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:54.940655 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:54.940656 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:54.940851 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:54.940940 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:54.941038 (+ 98us) server_negotiation.cc:299] Negotiation successful
0806 22:49:54.941115 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
WARNING: no policy specified for kudu/127.28.218.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.218.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.218.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.195.keytab.
Entry for principal kudu/127.28.218.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.195.keytab.
I20260806 22:49:54.966180 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056586788339-29547-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056586788339-29547-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056586788339-29547-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056586788339-29547-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.218.195:0
--local_ip_for_outbound_sockets=127.28.218.195
--webserver_interface=127.28.218.195
--webserver_port=0
--tserver_master_addrs=127.28.218.254:38323
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.195.keytab
--principal=kudu/127.28.218.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.218.212:41221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:49:55.046958 31706 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:49:55.047209 31706 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:49:55.047254 31706 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:49:55.048421 31706 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:49:55.048483 31706 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:49:55.048517 31706 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 22:49:55.048550 31706 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 22:49:55.048583 31706 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.218.195
I20260806 22:49:55.049989 31706 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:41221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056586788339-29547-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056586788339-29547-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.195.keytab
--principal=kudu/127.28.218.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.218.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056586788339-29547-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.218.195
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.28.218.254:38323
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31706
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.218.195
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056586788339-29547-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:49:55.050261 31706 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:49:55.050488 31706 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:49:55.055330 31712 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:55.055303 31711 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:55.055299 31714 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:49:55.055538 31706 server_base.cc:1034] running on GCE node
I20260806 22:49:55.055757 31706 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:49:55.055990 31706 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:49:55.057122 31706 hybrid_clock.cc:648] HybridClock initialized: now 1786056595057113 us; error 27 us; skew 500 ppm
Aug 06 22:49:55 dist-test-slave-1xrh krb5kdc[31289](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056595, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:49:55.064719 31706 init.cc:377] Logged in from keytab as kudu/127.28.218.195@KRBTEST.COM (short username kudu)
I20260806 22:49:55.065131 31706 webserver.cc:460] Webserver started at http://127.28.218.195:43993/ using document root <none> and password file <none>
I20260806 22:49:55.065361 31706 fs_manager.cc:359] Metadata directory not provided
I20260806 22:49:55.065428 31706 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:49:55.065583 31706 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:49:55.066569 31706 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056586788339-29547-0/minicluster-data/ts-2/data/instance:
uuid: "4eaa65e90f624ffaa081592d38dfb33f"
format_stamp: "Formatted at 2026-08-06 22:49:55 on dist-test-slave-1xrh"
server_key: "cbfb359cd4e4f5ebe2b953befaf4d741"
server_key_iv: "b3828614920d908f912324b12de06a0f"
server_key_version: "encryptionkey@0"
I20260806 22:49:55.066919 31706 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056586788339-29547-0/minicluster-data/ts-2/wal/instance:
uuid: "4eaa65e90f624ffaa081592d38dfb33f"
format_stamp: "Formatted at 2026-08-06 22:49:55 on dist-test-slave-1xrh"
server_key: "cbfb359cd4e4f5ebe2b953befaf4d741"
server_key_iv: "b3828614920d908f912324b12de06a0f"
server_key_version: "encryptionkey@0"
I20260806 22:49:55.068257 31706 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:49:55.069067 31721 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:55.069245 31706 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:49:55.069335 31706 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056586788339-29547-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056586788339-29547-0/minicluster-data/ts-2/wal
uuid: "4eaa65e90f624ffaa081592d38dfb33f"
format_stamp: "Formatted at 2026-08-06 22:49:55 on dist-test-slave-1xrh"
server_key: "cbfb359cd4e4f5ebe2b953befaf4d741"
server_key_iv: "b3828614920d908f912324b12de06a0f"
server_key_version: "encryptionkey@0"
I20260806 22:49:55.069401 31706 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056586788339-29547-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056586788339-29547-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056586788339-29547-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:49:55.085731 31706 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:49:55.087457 31706 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:49:55.087568 31706 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:49:55.087961 31706 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:49:55.088027 31706 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:55.088068 31706 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:49:55.088096 31706 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 22:49:55 dist-test-slave-1xrh krb5kdc[31289](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056595, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.195@KRBTEST.COM for kudu/127.28.218.254@KRBTEST.COM
I20260806 22:49:55.095163 31706 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.195:41453
I20260806 22:49:55.095233 31842 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.195:41453 every 8 connection(s)
I20260806 22:49:55.095508 31706 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056586788339-29547-0/minicluster-data/ts-2/data/info.pb
I20260806 22:49:55.098479 31400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:55.088451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.195:54017 (local address 127.28.218.254:38323)
0806 22:49:55.088526 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:55.088529 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:55.088681 (+ 152us) server_negotiation.cc:407] Connection header received
0806 22:49:55.089133 (+ 452us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:55.089134 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:55.089148 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:55.089233 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:55.090045 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:55.090882 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:55.092007 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:55.092215 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:55.092300 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:55.095520 (+ 3220us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:55.095522 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:55.095522 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:49:55.095530 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:55.097073 (+ 1543us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:55.097491 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:55.097492 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:55.097493 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:55.097557 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:55.097811 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:55.097812 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:55.097812 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:55.097934 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:55.098012 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:55.098293 (+ 281us) server_negotiation.cc:299] Negotiation successful
0806 22:49:55.098386 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 22:49:55.098630 31731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:55.088408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:38323 (local address 127.28.218.195:54017)
0806 22:49:55.088552 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:55.088594 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:55.089043 (+ 449us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:55.089279 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:55.089282 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:55.089658 (+ 376us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:49:55.089934 (+ 276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:55.089940 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:55.091014 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:55.091016 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:55.091902 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:55.091905 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:55.092324 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:55.092325 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:55.092456 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:55.092979 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:49:55.092991 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:49:55.095424 (+ 2433us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:49:55.097175 (+ 1751us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:55.097182 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:55.097184 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:55.097398 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:55.097623 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:55.097624 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:55.097624 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:55.097734 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:55.098003 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:49:55.098009 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:49:55.098156 (+ 147us) client_negotiation.cc:769] Sending connection context
0806 22:49:55.098256 (+ 100us) client_negotiation.cc:240] Negotiation successful
0806 22:49:55.098402 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":77,"threads_started":1}
I20260806 22:49:55.099116 31843 heartbeater.cc:347] Connected to a master server at 127.28.218.254:38323
I20260806 22:49:55.099248 31843 heartbeater.cc:464] Registering TS with master...
I20260806 22:49:55.099414 31843 heartbeater.cc:511] Master 127.28.218.254:38323 requested a full tablet report, sending...
I20260806 22:49:55.099787 31335 ts_manager.cc:194] Registered new tserver with Master: 4eaa65e90f624ffaa081592d38dfb33f (127.28.218.195:41453)
I20260806 22:49:55.100284 31335 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.218.195@KRBTEST.COM'} at 127.28.218.195:54017
I20260806 22:49:55.101060 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 31706
I20260806 22:49:55.101143 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056586788339-29547-0/minicluster-data/ts-2/wal/instance
I20260806 22:49:55.101332 29547 external_mini_cluster.cc:1428] Setting key e1d11fb6fecedfc1c8937994d0defd6b
I20260806 22:49:55.102097 29547 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:49:55.106273 31400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:55.100300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.195:51457 (local address 127.28.218.254:38323)
0806 22:49:55.100340 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:55.100341 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:55.100428 (+ 87us) server_negotiation.cc:407] Connection header received
0806 22:49:55.100490 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:55.100490 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:55.100499 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:55.100549 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:55.100876 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:55.101525 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:55.102495 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:55.102674 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:55.102726 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:55.103859 (+ 1133us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:55.103860 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:55.103860 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:49:55.103866 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:55.105213 (+ 1347us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:55.105616 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:55.105617 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:55.105617 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:55.105663 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:55.105857 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:55.105858 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:55.105858 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:55.105947 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:55.105996 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:55.106165 (+ 169us) server_negotiation.cc:299] Negotiation successful
0806 22:49:55.106219 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 22:49:55.106397 31849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:55.100226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:38323 (local address 127.28.218.195:51457)
0806 22:49:55.100367 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:55.100379 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:55.100441 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:55.100564 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:55.100564 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:55.100736 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:49:55.100796 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:55.100798 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:55.101618 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:55.101619 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:55.102414 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:55.102416 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:55.102756 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:55.102757 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:55.102847 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:55.103324 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:49:55.103330 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:49:55.103754 (+ 424us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:49:55.105295 (+ 1541us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:55.105296 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:55.105296 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:55.105543 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:55.105728 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:55.105729 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:55.105729 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:55.105798 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:55.106011 (+ 213us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:49:55.106012 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:49:55.106060 (+ 48us) client_negotiation.cc:769] Sending connection context
0806 22:49:55.106133 (+ 73us) client_negotiation.cc:240] Negotiation successful
0806 22:49:55.106243 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":81,"threads_started":1}
Aug 06 22:49:55 dist-test-slave-1xrh krb5kdc[31289](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056595, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 22:49:55 dist-test-slave-1xrh krb5kdc[31289](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056595, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.218.254@KRBTEST.COM
I20260806 22:49:55.126068 31400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:55.119567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32916 (local address 127.28.218.254:38323)
0806 22:49:55.119611 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:55.119613 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:55.119638 (+ 25us) server_negotiation.cc:407] Connection header received
0806 22:49:55.119689 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:55.119690 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:55.119697 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:55.119749 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:55.120280 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:55.120720 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:55.121411 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:55.121590 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:55.121653 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:55.123795 (+ 2142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:55.123796 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:55.123797 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:49:55.123812 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:55.125210 (+ 1398us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:55.125517 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:55.125518 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:55.125518 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:55.125560 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:55.125702 (+ 142us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:55.125703 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:55.125703 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:55.125796 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:55.125866 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:55.125949 (+ 83us) server_negotiation.cc:299] Negotiation successful
0806 22:49:55.126012 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260806 22:49:55.127741 31335 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32916:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:49:55.129290 31335 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:49:55.135766 31858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:55.131427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.195:41453 (local address 127.0.0.1:60258)
0806 22:49:55.131771 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:55.131780 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:55.131818 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:55.132015 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:55.132018 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:55.132021 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:49:55.132088 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:55.132090 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:55.133382 (+ 1292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:55.133384 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:55.134526 (+ 1142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:55.134529 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:55.135448 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:55.135449 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:55.135545 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:55.135560 (+ 15us) client_negotiation.cc:769] Sending connection context
0806 22:49:55.135639 (+ 79us) client_negotiation.cc:240] Negotiation successful
0806 22:49:55.135693 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":331,"mutex_wait_us":10,"thread_start_us":52,"threads_started":1}
I20260806 22:49:55.135941 31863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:55.131979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60708 (local address 127.28.218.194:33853)
0806 22:49:55.132213 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:55.132218 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:55.132228 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:49:55.132264 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:55.132266 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:55.132282 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:55.132361 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:49:55.132491 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:55.133250 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:55.134655 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:55.135427 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:55.135494 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:55.135548 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:55.135808 (+ 260us) server_negotiation.cc:299] Negotiation successful
0806 22:49:55.135843 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":204,"thread_start_us":115,"threads_started":1}
I20260806 22:49:55.135990 31859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:55.131413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.194:33853 (local address 127.0.0.1:60708)
0806 22:49:55.131526 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:55.131569 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:55.131656 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:55.132358 (+ 702us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:55.132363 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:55.132365 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:49:55.132406 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:55.132407 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:55.133332 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:55.133334 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:55.134562 (+ 1228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:55.134565 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:55.135584 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:55.135585 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:55.135648 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:55.135652 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:49:55.135704 (+ 52us) client_negotiation.cc:240] Negotiation successful
0806 22:49:55.135746 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":90,"mutex_wait_us":41,"thread_start_us":64,"threads_started":1}
I20260806 22:49:55.136118 31860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:55.131440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.193:35231 (local address 127.0.0.1:41908)
0806 22:49:55.131857 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:55.131869 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:55.131919 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:55.132636 (+ 717us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:55.132637 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:55.132639 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:49:55.132672 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:55.132673 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:55.133875 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:55.133876 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:55.134915 (+ 1039us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:55.134917 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:55.135894 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:55.135896 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:55.135956 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:55.135960 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:49:55.136012 (+ 52us) client_negotiation.cc:240] Negotiation successful
0806 22:49:55.136051 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":43,"threads_started":1}
I20260806 22:49:55.136395 31862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:55.131701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41908 (local address 127.28.218.193:35231)
0806 22:49:55.132486 (+ 785us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:55.132490 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:55.132499 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:49:55.132533 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:55.132535 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:55.132550 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:55.132625 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:49:55.132736 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:55.133777 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:55.135003 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:55.135754 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:55.135816 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:55.135862 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:55.136241 (+ 379us) server_negotiation.cc:299] Negotiation successful
0806 22:49:55.136280 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":750,"thread_start_us":86,"threads_started":1}
I20260806 22:49:55.135815 31861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:55.131677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60258 (local address 127.28.218.195:41453)
0806 22:49:55.131840 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:55.131844 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:55.131855 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:49:55.131895 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:55.131897 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:55.131913 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:55.131993 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:49:55.132182 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:55.133274 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:55.134638 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:55.135341 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:55.135448 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:55.135560 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:55.135653 (+ 93us) server_negotiation.cc:299] Negotiation successful
0806 22:49:55.135716 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":101,"threads_started":1}
I20260806 22:49:55.136767 31627 tablet_service.cc:1467] Processing CreateTablet for tablet 19a3c5d01ab344db8b6ca7a2faf14e45 (DEFAULT_TABLE table=test-table [id=eecf0e05786942198649dc4b96bf651d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:49:55.137115 31627 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19a3c5d01ab344db8b6ca7a2faf14e45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:49:55.139690 31478 tablet_service.cc:1467] Processing CreateTablet for tablet 19a3c5d01ab344db8b6ca7a2faf14e45 (DEFAULT_TABLE table=test-table [id=eecf0e05786942198649dc4b96bf651d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:49:55.139784 31774 tablet_service.cc:1467] Processing CreateTablet for tablet 19a3c5d01ab344db8b6ca7a2faf14e45 (DEFAULT_TABLE table=test-table [id=eecf0e05786942198649dc4b96bf651d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:49:55.140053 31478 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19a3c5d01ab344db8b6ca7a2faf14e45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:49:55.140097 31774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19a3c5d01ab344db8b6ca7a2faf14e45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:49:55.140100 31866 tablet_bootstrap.cc:492] T 19a3c5d01ab344db8b6ca7a2faf14e45 P 8da8827f19c842f6bfae8a9ee27ef69a: Bootstrap starting.
I20260806 22:49:55.140726 31866 tablet_bootstrap.cc:654] T 19a3c5d01ab344db8b6ca7a2faf14e45 P 8da8827f19c842f6bfae8a9ee27ef69a: Neither blocks nor log segments found. Creating new log.
I20260806 22:49:55.141095 31866 log.cc:824] T 19a3c5d01ab344db8b6ca7a2faf14e45 P 8da8827f19c842f6bfae8a9ee27ef69a: Log is configured to *not* fsync() on all Append() calls
I20260806 22:49:55.141888 31866 tablet_bootstrap.cc:492] T 19a3c5d01ab344db8b6ca7a2faf14e45 P 8da8827f19c842f6bfae8a9ee27ef69a: No bootstrap required, opened a new log
I20260806 22:49:55.141947 31866 ts_tablet_manager.cc:1397] T 19a3c5d01ab344db8b6ca7a2faf14e45 P 8da8827f19c842f6bfae8a9ee27ef69a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:49:55.142455 31868 tablet_bootstrap.cc:492] T 19a3c5d01ab344db8b6ca7a2faf14e45 P 4eaa65e90f624ffaa081592d38dfb33f: Bootstrap starting.
I20260806 22:49:55.143287 31868 tablet_bootstrap.cc:654] T 19a3c5d01ab344db8b6ca7a2faf14e45 P 4eaa65e90f624ffaa081592d38dfb33f: Neither blocks nor log segments found. Creating new log.
I20260806 22:49:55.143579 31868 log.cc:824] T 19a3c5d01ab344db8b6ca7a2faf14e45 P 4eaa65e90f624ffaa081592d38dfb33f: Log is configured to *not* fsync() on all Append() calls
I20260806 22:49:55.143894 31870 tablet_bootstrap.cc:492] T 19a3c5d01ab344db8b6ca7a2faf14e45 P 9edf436137414f54b26af5a8ec47d81c: Bootstrap starting.
I20260806 22:49:55.144380 31870 tablet_bootstrap.cc:654] T 19a3c5d01ab344db8b6ca7a2faf14e45 P 9edf436137414f54b26af5a8ec47d81c: Neither blocks nor log segments found. Creating new log.
I20260806 22:49:55.144328 31866 raft_consensus.cc:348] T 19a3c5d01ab344db8b6ca7a2faf14e45 P 8da8827f19c842f6bfae8a9ee27ef69a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8da8827f19c842f6bfae8a9ee27ef69a" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 33853 } } peers { permanent_uuid: "4eaa65e90f624ffaa081592d38dfb33f" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 41453 } } peers { permanent_uuid: "9edf436137414f54b26af5a8ec47d81c" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 35231 } }
I20260806 22:49:55.144448 31866 raft_consensus.cc:374] T 19a3c5d01ab344db8b6ca7a2faf14e45 P 8da8827f19c842f6bfae8a9ee27ef69a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:49:55.144512 31866 raft_consensus.cc:729] T 19a3c5d01ab344db8b6ca7a2faf14e45 P 8da8827f19c842f6bfae8a9ee27ef69a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8da8827f19c842f6bfae8a9ee27ef69a, State: Initialized, Role: FOLLOWER
I20260806 22:49:55.144634 31866 consensus_queue.cc:260] T 19a3c5d01ab344db8b6ca7a2faf14e45 P 8da8827f19c842f6bfae8a9ee27ef69a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8da8827f19c842f6bfae8a9ee27ef69a" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 33853 } } peers { permanent_uuid: "4eaa65e90f624ffaa081592d38dfb33f" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 41453 } } peers { permanent_uuid: "9edf436137414f54b26af5a8ec47d81c" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 35231 } }
I20260806 22:49:55.144752 31870 log.cc:824] T 19a3c5d01ab344db8b6ca7a2faf14e45 P 9edf436137414f54b26af5a8ec47d81c: Log is configured to *not* fsync() on all Append() calls
I20260806 22:49:55.144887 31866 ts_tablet_manager.cc:1428] T 19a3c5d01ab344db8b6ca7a2faf14e45 P 8da8827f19c842f6bfae8a9ee27ef69a: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260806 22:49:55.144972 31697 heartbeater.cc:503] Master 127.28.218.254:38323 was elected leader, sending a full tablet report...
I20260806 22:49:55.145476 31868 tablet_bootstrap.cc:492] T 19a3c5d01ab344db8b6ca7a2faf14e45 P 4eaa65e90f624ffaa081592d38dfb33f: No bootstrap required, opened a new log
I20260806 22:49:55.145484 31870 tablet_bootstrap.cc:492] T 19a3c5d01ab344db8b6ca7a2faf14e45 P 9edf436137414f54b26af5a8ec47d81c: No bootstrap required, opened a new log
I20260806 22:49:55.145550 31868 ts_tablet_manager.cc:1397] T 19a3c5d01ab344db8b6ca7a2faf14e45 P 4eaa65e90f624ffaa081592d38dfb33f: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260806 22:49:55.145618 31870 ts_tablet_manager.cc:1397] T 19a3c5d01ab344db8b6ca7a2faf14e45 P 9edf436137414f54b26af5a8ec47d81c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:49:55.147560 31868 raft_consensus.cc:348] T 19a3c5d01ab344db8b6ca7a2faf14e45 P 4eaa65e90f624ffaa081592d38dfb33f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8da8827f19c842f6bfae8a9ee27ef69a" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 33853 } } peers { permanent_uuid: "4eaa65e90f624ffaa081592d38dfb33f" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 41453 } } peers { permanent_uuid: "9edf436137414f54b26af5a8ec47d81c" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 35231 } }
I20260806 22:49:55.147588 31870 raft_consensus.cc:348] T 19a3c5d01ab344db8b6ca7a2faf14e45 P 9edf436137414f54b26af5a8ec47d81c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8da8827f19c842f6bfae8a9ee27ef69a" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 33853 } } peers { permanent_uuid: "4eaa65e90f624ffaa081592d38dfb33f" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 41453 } } peers { permanent_uuid: "9edf436137414f54b26af5a8ec47d81c" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 35231 } }
I20260806 22:49:55.147717 31868 raft_consensus.cc:374] T 19a3c5d01ab344db8b6ca7a2faf14e45 P 4eaa65e90f624ffaa081592d38dfb33f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:49:55.147717 31870 raft_consensus.cc:374] T 19a3c5d01ab344db8b6ca7a2faf14e45 P 9edf436137414f54b26af5a8ec47d81c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:49:55.147753 31870 raft_consensus.cc:729] T 19a3c5d01ab344db8b6ca7a2faf14e45 P 9edf436137414f54b26af5a8ec47d81c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9edf436137414f54b26af5a8ec47d81c, State: Initialized, Role: FOLLOWER
I20260806 22:49:55.147753 31868 raft_consensus.cc:729] T 19a3c5d01ab344db8b6ca7a2faf14e45 P 4eaa65e90f624ffaa081592d38dfb33f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4eaa65e90f624ffaa081592d38dfb33f, State: Initialized, Role: FOLLOWER
I20260806 22:49:55.147869 31868 consensus_queue.cc:260] T 19a3c5d01ab344db8b6ca7a2faf14e45 P 4eaa65e90f624ffaa081592d38dfb33f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8da8827f19c842f6bfae8a9ee27ef69a" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 33853 } } peers { permanent_uuid: "4eaa65e90f624ffaa081592d38dfb33f" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 41453 } } peers { permanent_uuid: "9edf436137414f54b26af5a8ec47d81c" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 35231 } }
I20260806 22:49:55.147869 31870 consensus_queue.cc:260] T 19a3c5d01ab344db8b6ca7a2faf14e45 P 9edf436137414f54b26af5a8ec47d81c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8da8827f19c842f6bfae8a9ee27ef69a" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 33853 } } peers { permanent_uuid: "4eaa65e90f624ffaa081592d38dfb33f" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 41453 } } peers { permanent_uuid: "9edf436137414f54b26af5a8ec47d81c" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 35231 } }
I20260806 22:49:55.148118 31870 ts_tablet_manager.cc:1428] T 19a3c5d01ab344db8b6ca7a2faf14e45 P 9edf436137414f54b26af5a8ec47d81c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:49:55.148118 31868 ts_tablet_manager.cc:1428] T 19a3c5d01ab344db8b6ca7a2faf14e45 P 4eaa65e90f624ffaa081592d38dfb33f: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 22:49:55.148157 31843 heartbeater.cc:503] Master 127.28.218.254:38323 was elected leader, sending a full tablet report...
I20260806 22:49:55.148424 31550 heartbeater.cc:503] Master 127.28.218.254:38323 was elected leader, sending a full tablet report...
W20260806 22:49:55.181234 31698 tablet.cc:2369] T 19a3c5d01ab344db8b6ca7a2faf14e45 P 8da8827f19c842f6bfae8a9ee27ef69a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 22:49:55.280612 31551 tablet.cc:2369] T 19a3c5d01ab344db8b6ca7a2faf14e45 P 9edf436137414f54b26af5a8ec47d81c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 22:49:55.346469 31844 tablet.cc:2369] T 19a3c5d01ab344db8b6ca7a2faf14e45 P 4eaa65e90f624ffaa081592d38dfb33f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:49:55.375315 31872 raft_consensus.cc:482] T 19a3c5d01ab344db8b6ca7a2faf14e45 P 8da8827f19c842f6bfae8a9ee27ef69a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:49:55.375495 31872 raft_consensus.cc:504] T 19a3c5d01ab344db8b6ca7a2faf14e45 P 8da8827f19c842f6bfae8a9ee27ef69a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8da8827f19c842f6bfae8a9ee27ef69a" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 33853 } } peers { permanent_uuid: "4eaa65e90f624ffaa081592d38dfb33f" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 41453 } } peers { permanent_uuid: "9edf436137414f54b26af5a8ec47d81c" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 35231 } }
I20260806 22:49:55.375916 31872 leader_election.cc:288] T 19a3c5d01ab344db8b6ca7a2faf14e45 P 8da8827f19c842f6bfae8a9ee27ef69a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4eaa65e90f624ffaa081592d38dfb33f (127.28.218.195:41453), 9edf436137414f54b26af5a8ec47d81c (127.28.218.193:35231)
I20260806 22:49:55.378696 31862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:55.376168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.194:33127 (local address 127.28.218.193:35231)
0806 22:49:55.376220 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:55.376222 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:55.376386 (+ 164us) server_negotiation.cc:407] Connection header received
0806 22:49:55.376425 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:55.376425 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:55.376435 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:55.376494 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:49:55.376674 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:55.377107 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:55.378012 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:55.378339 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:55.378393 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:55.378443 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:55.378579 (+ 136us) server_negotiation.cc:299] Negotiation successful
0806 22:49:55.378625 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260806 22:49:55.378708 31876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:55.376121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.193:35231 (local address 127.28.218.194:33127)
0806 22:49:55.376305 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:55.376316 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:55.376373 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:55.376507 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:55.376508 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:55.376509 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:49:55.376605 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:55.376607 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:55.377214 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:55.377214 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:55.377899 (+ 685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:55.377901 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:55.378422 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:55.378423 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:55.378510 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:55.378513 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:49:55.378555 (+ 42us) client_negotiation.cc:240] Negotiation successful
0806 22:49:55.378582 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"mutex_wait_us":18,"thread_start_us":111,"threads_started":1}
I20260806 22:49:55.378746 31861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:55.376208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.194:41061 (local address 127.28.218.195:41453)
0806 22:49:55.376259 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:55.376261 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:55.376270 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:49:55.376302 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:55.376303 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:55.376313 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:55.376379 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:49:55.376554 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:55.377090 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:55.377989 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:55.378345 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:55.378430 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:55.378439 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:55.378635 (+ 196us) server_negotiation.cc:299] Negotiation successful
0806 22:49:55.378675 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260806 22:49:55.378713 31583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:55.376110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.195:41453 (local address 127.28.218.194:41061)
0806 22:49:55.376165 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:55.376179 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:55.376229 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:55.376400 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:55.376401 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:55.376403 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:49:55.376479 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:55.376481 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:55.377173 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:55.377175 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:55.377881 (+ 706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:55.377883 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:55.378460 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:55.378461 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:55.378536 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:55.378556 (+ 20us) client_negotiation.cc:769] Sending connection context
0806 22:49:55.378613 (+ 57us) client_negotiation.cc:240] Negotiation successful
0806 22:49:55.378644 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260806 22:49:55.379175 31501 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19a3c5d01ab344db8b6ca7a2faf14e45" candidate_uuid: "8da8827f19c842f6bfae8a9ee27ef69a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9edf436137414f54b26af5a8ec47d81c" is_pre_election: true
I20260806 22:49:55.379339 31501 raft_consensus.cc:2454] T 19a3c5d01ab344db8b6ca7a2faf14e45 P 9edf436137414f54b26af5a8ec47d81c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8da8827f19c842f6bfae8a9ee27ef69a in term 0.
I20260806 22:49:55.379362 31795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19a3c5d01ab344db8b6ca7a2faf14e45" candidate_uuid: "8da8827f19c842f6bfae8a9ee27ef69a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4eaa65e90f624ffaa081592d38dfb33f" is_pre_election: true
I20260806 22:49:55.379498 31795 raft_consensus.cc:2454] T 19a3c5d01ab344db8b6ca7a2faf14e45 P 4eaa65e90f624ffaa081592d38dfb33f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8da8827f19c842f6bfae8a9ee27ef69a in term 0.
I20260806 22:49:55.379563 31576 leader_election.cc:302] T 19a3c5d01ab344db8b6ca7a2faf14e45 P 8da8827f19c842f6bfae8a9ee27ef69a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8da8827f19c842f6bfae8a9ee27ef69a, 9edf436137414f54b26af5a8ec47d81c; no voters:
I20260806 22:49:55.379688 31872 raft_consensus.cc:2781] T 19a3c5d01ab344db8b6ca7a2faf14e45 P 8da8827f19c842f6bfae8a9ee27ef69a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:49:55.379734 31872 raft_consensus.cc:482] T 19a3c5d01ab344db8b6ca7a2faf14e45 P 8da8827f19c842f6bfae8a9ee27ef69a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:49:55.379756 31872 raft_consensus.cc:3037] T 19a3c5d01ab344db8b6ca7a2faf14e45 P 8da8827f19c842f6bfae8a9ee27ef69a [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:49:55.380765 31872 raft_consensus.cc:504] T 19a3c5d01ab344db8b6ca7a2faf14e45 P 8da8827f19c842f6bfae8a9ee27ef69a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8da8827f19c842f6bfae8a9ee27ef69a" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 33853 } } peers { permanent_uuid: "4eaa65e90f624ffaa081592d38dfb33f" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 41453 } } peers { permanent_uuid: "9edf436137414f54b26af5a8ec47d81c" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 35231 } }
I20260806 22:49:55.380897 31872 leader_election.cc:288] T 19a3c5d01ab344db8b6ca7a2faf14e45 P 8da8827f19c842f6bfae8a9ee27ef69a [CANDIDATE]: Term 1 election: Requested vote from peers 4eaa65e90f624ffaa081592d38dfb33f (127.28.218.195:41453), 9edf436137414f54b26af5a8ec47d81c (127.28.218.193:35231)
I20260806 22:49:55.381045 31795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19a3c5d01ab344db8b6ca7a2faf14e45" candidate_uuid: "8da8827f19c842f6bfae8a9ee27ef69a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4eaa65e90f624ffaa081592d38dfb33f"
I20260806 22:49:55.381076 31501 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19a3c5d01ab344db8b6ca7a2faf14e45" candidate_uuid: "8da8827f19c842f6bfae8a9ee27ef69a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9edf436137414f54b26af5a8ec47d81c"
I20260806 22:49:55.381129 31795 raft_consensus.cc:3037] T 19a3c5d01ab344db8b6ca7a2faf14e45 P 4eaa65e90f624ffaa081592d38dfb33f [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:49:55.381139 31501 raft_consensus.cc:3037] T 19a3c5d01ab344db8b6ca7a2faf14e45 P 9edf436137414f54b26af5a8ec47d81c [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:49:55.382020 31795 raft_consensus.cc:2454] T 19a3c5d01ab344db8b6ca7a2faf14e45 P 4eaa65e90f624ffaa081592d38dfb33f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8da8827f19c842f6bfae8a9ee27ef69a in term 1.
I20260806 22:49:55.382081 31501 raft_consensus.cc:2454] T 19a3c5d01ab344db8b6ca7a2faf14e45 P 9edf436137414f54b26af5a8ec47d81c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8da8827f19c842f6bfae8a9ee27ef69a in term 1.
I20260806 22:49:55.382213 31578 leader_election.cc:302] T 19a3c5d01ab344db8b6ca7a2faf14e45 P 8da8827f19c842f6bfae8a9ee27ef69a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4eaa65e90f624ffaa081592d38dfb33f, 8da8827f19c842f6bfae8a9ee27ef69a; no voters:
I20260806 22:49:55.382303 31872 raft_consensus.cc:2781] T 19a3c5d01ab344db8b6ca7a2faf14e45 P 8da8827f19c842f6bfae8a9ee27ef69a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:49:55.382475 31872 raft_consensus.cc:686] T 19a3c5d01ab344db8b6ca7a2faf14e45 P 8da8827f19c842f6bfae8a9ee27ef69a [term 1 LEADER]: Becoming Leader. State: Replica: 8da8827f19c842f6bfae8a9ee27ef69a, State: Running, Role: LEADER
I20260806 22:49:55.382572 31872 consensus_queue.cc:237] T 19a3c5d01ab344db8b6ca7a2faf14e45 P 8da8827f19c842f6bfae8a9ee27ef69a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8da8827f19c842f6bfae8a9ee27ef69a" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 33853 } } peers { permanent_uuid: "4eaa65e90f624ffaa081592d38dfb33f" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 41453 } } peers { permanent_uuid: "9edf436137414f54b26af5a8ec47d81c" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 35231 } }
I20260806 22:49:55.383430 31335 catalog_manager.cc:5495] T 19a3c5d01ab344db8b6ca7a2faf14e45 P 8da8827f19c842f6bfae8a9ee27ef69a reported cstate change: term changed from 0 to 1, leader changed from <none> to 8da8827f19c842f6bfae8a9ee27ef69a (127.28.218.194). New cstate: current_term: 1 leader_uuid: "8da8827f19c842f6bfae8a9ee27ef69a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8da8827f19c842f6bfae8a9ee27ef69a" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 33853 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4eaa65e90f624ffaa081592d38dfb33f" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 41453 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9edf436137414f54b26af5a8ec47d81c" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 35231 } health_report { overall_health: UNKNOWN } } }
Aug 06 22:49:55 dist-test-slave-1xrh krb5kdc[31289](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056594, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.254@KRBTEST.COM for kudu/127.28.218.254@KRBTEST.COM
I20260806 22:49:55.434325 31400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:55.427739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32918 (local address 127.28.218.254:38323)
0806 22:49:55.427827 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:55.427829 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:55.427891 (+ 62us) server_negotiation.cc:407] Connection header received
0806 22:49:55.427933 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:55.427934 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:55.427942 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:55.427993 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:55.428396 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:55.428867 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:55.429533 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:55.429695 (+ 162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:55.429740 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:55.431669 (+ 1929us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:55.431670 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:55.431675 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:49:55.431681 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:55.433076 (+ 1395us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:55.433456 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:55.433457 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:55.433457 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:55.433515 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:55.433707 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:55.433708 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:55.433708 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:55.433812 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:55.433890 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:55.434051 (+ 161us) server_negotiation.cc:299] Negotiation successful
0806 22:49:55.434165 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":59}
I20260806 22:49:55.434378 31884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:55.427671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:38323 (local address 127.0.0.1:32918)
0806 22:49:55.427824 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:55.427833 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:55.427889 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:55.428008 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:55.428008 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:55.428292 (+ 284us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:49:55.428335 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:55.428337 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:55.428969 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:55.428970 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:55.429471 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:55.429472 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:55.429751 (+ 279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:55.429752 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:55.429789 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:55.430231 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:49:55.430235 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:49:55.431574 (+ 1339us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:49:55.433178 (+ 1604us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:55.433185 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:55.433188 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:55.433380 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:55.433586 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:55.433587 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:55.433587 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:55.433648 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:55.433893 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:49:55.433901 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:49:55.433943 (+ 42us) client_negotiation.cc:769] Sending connection context
0806 22:49:55.434028 (+ 85us) client_negotiation.cc:240] Negotiation successful
0806 22:49:55.434241 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"mutex_wait_us":97,"thread_start_us":70,"threads_started":1}
I20260806 22:49:55.436093 31335 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.28.218.254@KRBTEST.COM'} at 127.0.0.1:32918:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 22:49:55.436269 31335 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:49:55.437652 31627 tablet_service.cc:1467] Processing CreateTablet for tablet 8dd1b555d13a4c64b5914f580ea74917 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=9e6376d5b5b241449e9690c406ce1469]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 22:49:55.437700 31774 tablet_service.cc:1467] Processing CreateTablet for tablet 8dd1b555d13a4c64b5914f580ea74917 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=9e6376d5b5b241449e9690c406ce1469]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 22:49:55.437830 31627 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8dd1b555d13a4c64b5914f580ea74917. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:49:55.437876 31774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8dd1b555d13a4c64b5914f580ea74917. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:49:55.439092 31868 tablet_bootstrap.cc:492] T 8dd1b555d13a4c64b5914f580ea74917 P 4eaa65e90f624ffaa081592d38dfb33f: Bootstrap starting.
I20260806 22:49:55.439217 31478 tablet_service.cc:1467] Processing CreateTablet for tablet 8dd1b555d13a4c64b5914f580ea74917 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=9e6376d5b5b241449e9690c406ce1469]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 22:49:55.439388 31478 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8dd1b555d13a4c64b5914f580ea74917. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:49:55.439709 31868 tablet_bootstrap.cc:654] T 8dd1b555d13a4c64b5914f580ea74917 P 4eaa65e90f624ffaa081592d38dfb33f: Neither blocks nor log segments found. Creating new log.
I20260806 22:49:55.440356 31868 tablet_bootstrap.cc:492] T 8dd1b555d13a4c64b5914f580ea74917 P 4eaa65e90f624ffaa081592d38dfb33f: No bootstrap required, opened a new log
I20260806 22:49:55.440407 31868 ts_tablet_manager.cc:1397] T 8dd1b555d13a4c64b5914f580ea74917 P 4eaa65e90f624ffaa081592d38dfb33f: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 22:49:55.440541 31868 raft_consensus.cc:348] T 8dd1b555d13a4c64b5914f580ea74917 P 4eaa65e90f624ffaa081592d38dfb33f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9edf436137414f54b26af5a8ec47d81c" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 35231 } } peers { permanent_uuid: "8da8827f19c842f6bfae8a9ee27ef69a" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 33853 } } peers { permanent_uuid: "4eaa65e90f624ffaa081592d38dfb33f" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 41453 } }
I20260806 22:49:55.440609 31868 raft_consensus.cc:374] T 8dd1b555d13a4c64b5914f580ea74917 P 4eaa65e90f624ffaa081592d38dfb33f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:49:55.440634 31868 raft_consensus.cc:729] T 8dd1b555d13a4c64b5914f580ea74917 P 4eaa65e90f624ffaa081592d38dfb33f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4eaa65e90f624ffaa081592d38dfb33f, State: Initialized, Role: FOLLOWER
I20260806 22:49:55.440675 31868 consensus_queue.cc:260] T 8dd1b555d13a4c64b5914f580ea74917 P 4eaa65e90f624ffaa081592d38dfb33f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9edf436137414f54b26af5a8ec47d81c" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 35231 } } peers { permanent_uuid: "8da8827f19c842f6bfae8a9ee27ef69a" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 33853 } } peers { permanent_uuid: "4eaa65e90f624ffaa081592d38dfb33f" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 41453 } }
I20260806 22:49:55.440783 31868 ts_tablet_manager.cc:1428] T 8dd1b555d13a4c64b5914f580ea74917 P 4eaa65e90f624ffaa081592d38dfb33f: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:55.440865 31875 tablet_replica.cc:439] T 8dd1b555d13a4c64b5914f580ea74917 P 4eaa65e90f624ffaa081592d38dfb33f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9edf436137414f54b26af5a8ec47d81c" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 35231 } } peers { permanent_uuid: "8da8827f19c842f6bfae8a9ee27ef69a" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 33853 } } peers { permanent_uuid: "4eaa65e90f624ffaa081592d38dfb33f" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 41453 } } }
I20260806 22:49:55.440943 31870 tablet_bootstrap.cc:492] T 8dd1b555d13a4c64b5914f580ea74917 P 9edf436137414f54b26af5a8ec47d81c: Bootstrap starting.
I20260806 22:49:55.440975 31866 tablet_bootstrap.cc:492] T 8dd1b555d13a4c64b5914f580ea74917 P 8da8827f19c842f6bfae8a9ee27ef69a: Bootstrap starting.
I20260806 22:49:55.440970 31875 tablet_replica.cc:442] T 8dd1b555d13a4c64b5914f580ea74917 P 4eaa65e90f624ffaa081592d38dfb33f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:49:55.441331 31870 tablet_bootstrap.cc:654] T 8dd1b555d13a4c64b5914f580ea74917 P 9edf436137414f54b26af5a8ec47d81c: Neither blocks nor log segments found. Creating new log.
I20260806 22:49:55.441373 31866 tablet_bootstrap.cc:654] T 8dd1b555d13a4c64b5914f580ea74917 P 8da8827f19c842f6bfae8a9ee27ef69a: Neither blocks nor log segments found. Creating new log.
I20260806 22:49:55.442097 31870 tablet_bootstrap.cc:492] T 8dd1b555d13a4c64b5914f580ea74917 P 9edf436137414f54b26af5a8ec47d81c: No bootstrap required, opened a new log
I20260806 22:49:55.442157 31870 ts_tablet_manager.cc:1397] T 8dd1b555d13a4c64b5914f580ea74917 P 9edf436137414f54b26af5a8ec47d81c: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 22:49:55.442308 31870 raft_consensus.cc:348] T 8dd1b555d13a4c64b5914f580ea74917 P 9edf436137414f54b26af5a8ec47d81c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9edf436137414f54b26af5a8ec47d81c" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 35231 } } peers { permanent_uuid: "8da8827f19c842f6bfae8a9ee27ef69a" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 33853 } } peers { permanent_uuid: "4eaa65e90f624ffaa081592d38dfb33f" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 41453 } }
I20260806 22:49:55.442376 31870 raft_consensus.cc:374] T 8dd1b555d13a4c64b5914f580ea74917 P 9edf436137414f54b26af5a8ec47d81c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:49:55.442399 31870 raft_consensus.cc:729] T 8dd1b555d13a4c64b5914f580ea74917 P 9edf436137414f54b26af5a8ec47d81c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9edf436137414f54b26af5a8ec47d81c, State: Initialized, Role: FOLLOWER
I20260806 22:49:55.442448 31870 consensus_queue.cc:260] T 8dd1b555d13a4c64b5914f580ea74917 P 9edf436137414f54b26af5a8ec47d81c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9edf436137414f54b26af5a8ec47d81c" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 35231 } } peers { permanent_uuid: "8da8827f19c842f6bfae8a9ee27ef69a" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 33853 } } peers { permanent_uuid: "4eaa65e90f624ffaa081592d38dfb33f" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 41453 } }
I20260806 22:49:55.442580 31870 ts_tablet_manager.cc:1428] T 8dd1b555d13a4c64b5914f580ea74917 P 9edf436137414f54b26af5a8ec47d81c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:55.442632 31874 tablet_replica.cc:439] T 8dd1b555d13a4c64b5914f580ea74917 P 9edf436137414f54b26af5a8ec47d81c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9edf436137414f54b26af5a8ec47d81c" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 35231 } } peers { permanent_uuid: "8da8827f19c842f6bfae8a9ee27ef69a" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 33853 } } peers { permanent_uuid: "4eaa65e90f624ffaa081592d38dfb33f" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 41453 } } }
I20260806 22:49:55.442757 31874 tablet_replica.cc:442] T 8dd1b555d13a4c64b5914f580ea74917 P 9edf436137414f54b26af5a8ec47d81c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:49:55.442977 31866 tablet_bootstrap.cc:492] T 8dd1b555d13a4c64b5914f580ea74917 P 8da8827f19c842f6bfae8a9ee27ef69a: No bootstrap required, opened a new log
I20260806 22:49:55.443019 31866 ts_tablet_manager.cc:1397] T 8dd1b555d13a4c64b5914f580ea74917 P 8da8827f19c842f6bfae8a9ee27ef69a: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260806 22:49:55.443172 31866 raft_consensus.cc:348] T 8dd1b555d13a4c64b5914f580ea74917 P 8da8827f19c842f6bfae8a9ee27ef69a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9edf436137414f54b26af5a8ec47d81c" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 35231 } } peers { permanent_uuid: "8da8827f19c842f6bfae8a9ee27ef69a" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 33853 } } peers { permanent_uuid: "4eaa65e90f624ffaa081592d38dfb33f" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 41453 } }
I20260806 22:49:55.443233 31866 raft_consensus.cc:374] T 8dd1b555d13a4c64b5914f580ea74917 P 8da8827f19c842f6bfae8a9ee27ef69a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:49:55.443270 31866 raft_consensus.cc:729] T 8dd1b555d13a4c64b5914f580ea74917 P 8da8827f19c842f6bfae8a9ee27ef69a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8da8827f19c842f6bfae8a9ee27ef69a, State: Initialized, Role: FOLLOWER
I20260806 22:49:55.443320 31866 consensus_queue.cc:260] T 8dd1b555d13a4c64b5914f580ea74917 P 8da8827f19c842f6bfae8a9ee27ef69a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9edf436137414f54b26af5a8ec47d81c" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 35231 } } peers { permanent_uuid: "8da8827f19c842f6bfae8a9ee27ef69a" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 33853 } } peers { permanent_uuid: "4eaa65e90f624ffaa081592d38dfb33f" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 41453 } }
I20260806 22:49:55.443406 31866 ts_tablet_manager.cc:1428] T 8dd1b555d13a4c64b5914f580ea74917 P 8da8827f19c842f6bfae8a9ee27ef69a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:55.443459 31872 tablet_replica.cc:439] T 8dd1b555d13a4c64b5914f580ea74917 P 8da8827f19c842f6bfae8a9ee27ef69a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9edf436137414f54b26af5a8ec47d81c" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 35231 } } peers { permanent_uuid: "8da8827f19c842f6bfae8a9ee27ef69a" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 33853 } } peers { permanent_uuid: "4eaa65e90f624ffaa081592d38dfb33f" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 41453 } } }
I20260806 22:49:55.443549 31872 tablet_replica.cc:442] T 8dd1b555d13a4c64b5914f580ea74917 P 8da8827f19c842f6bfae8a9ee27ef69a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:49:55.633008 31872 raft_consensus.cc:482] T 8dd1b555d13a4c64b5914f580ea74917 P 8da8827f19c842f6bfae8a9ee27ef69a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:49:55.633100 31872 raft_consensus.cc:504] T 8dd1b555d13a4c64b5914f580ea74917 P 8da8827f19c842f6bfae8a9ee27ef69a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9edf436137414f54b26af5a8ec47d81c" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 35231 } } peers { permanent_uuid: "8da8827f19c842f6bfae8a9ee27ef69a" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 33853 } } peers { permanent_uuid: "4eaa65e90f624ffaa081592d38dfb33f" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 41453 } }
I20260806 22:49:55.633284 31872 leader_election.cc:288] T 8dd1b555d13a4c64b5914f580ea74917 P 8da8827f19c842f6bfae8a9ee27ef69a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9edf436137414f54b26af5a8ec47d81c (127.28.218.193:35231), 4eaa65e90f624ffaa081592d38dfb33f (127.28.218.195:41453)
I20260806 22:49:55.633486 31501 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8dd1b555d13a4c64b5914f580ea74917" candidate_uuid: "8da8827f19c842f6bfae8a9ee27ef69a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9edf436137414f54b26af5a8ec47d81c" is_pre_election: true
I20260806 22:49:55.633486 31795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8dd1b555d13a4c64b5914f580ea74917" candidate_uuid: "8da8827f19c842f6bfae8a9ee27ef69a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4eaa65e90f624ffaa081592d38dfb33f" is_pre_election: true
I20260806 22:49:55.633597 31501 raft_consensus.cc:2454] T 8dd1b555d13a4c64b5914f580ea74917 P 9edf436137414f54b26af5a8ec47d81c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8da8827f19c842f6bfae8a9ee27ef69a in term 0.
I20260806 22:49:55.633597 31795 raft_consensus.cc:2454] T 8dd1b555d13a4c64b5914f580ea74917 P 4eaa65e90f624ffaa081592d38dfb33f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8da8827f19c842f6bfae8a9ee27ef69a in term 0.
I20260806 22:49:55.633750 31576 leader_election.cc:302] T 8dd1b555d13a4c64b5914f580ea74917 P 8da8827f19c842f6bfae8a9ee27ef69a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8da8827f19c842f6bfae8a9ee27ef69a, 9edf436137414f54b26af5a8ec47d81c; no voters:
I20260806 22:49:55.633870 31872 raft_consensus.cc:2781] T 8dd1b555d13a4c64b5914f580ea74917 P 8da8827f19c842f6bfae8a9ee27ef69a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:49:55.633940 31872 raft_consensus.cc:482] T 8dd1b555d13a4c64b5914f580ea74917 P 8da8827f19c842f6bfae8a9ee27ef69a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:49:55.633980 31872 raft_consensus.cc:3037] T 8dd1b555d13a4c64b5914f580ea74917 P 8da8827f19c842f6bfae8a9ee27ef69a [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:49:55.634712 31872 raft_consensus.cc:504] T 8dd1b555d13a4c64b5914f580ea74917 P 8da8827f19c842f6bfae8a9ee27ef69a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9edf436137414f54b26af5a8ec47d81c" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 35231 } } peers { permanent_uuid: "8da8827f19c842f6bfae8a9ee27ef69a" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 33853 } } peers { permanent_uuid: "4eaa65e90f624ffaa081592d38dfb33f" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 41453 } }
I20260806 22:49:55.634856 31872 leader_election.cc:288] T 8dd1b555d13a4c64b5914f580ea74917 P 8da8827f19c842f6bfae8a9ee27ef69a [CANDIDATE]: Term 1 election: Requested vote from peers 9edf436137414f54b26af5a8ec47d81c (127.28.218.193:35231), 4eaa65e90f624ffaa081592d38dfb33f (127.28.218.195:41453)
I20260806 22:49:55.635022 31795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8dd1b555d13a4c64b5914f580ea74917" candidate_uuid: "8da8827f19c842f6bfae8a9ee27ef69a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4eaa65e90f624ffaa081592d38dfb33f"
I20260806 22:49:55.635026 31501 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8dd1b555d13a4c64b5914f580ea74917" candidate_uuid: "8da8827f19c842f6bfae8a9ee27ef69a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9edf436137414f54b26af5a8ec47d81c"
I20260806 22:49:55.635103 31501 raft_consensus.cc:3037] T 8dd1b555d13a4c64b5914f580ea74917 P 9edf436137414f54b26af5a8ec47d81c [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:49:55.635121 31795 raft_consensus.cc:3037] T 8dd1b555d13a4c64b5914f580ea74917 P 4eaa65e90f624ffaa081592d38dfb33f [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:49:55.635809 31501 raft_consensus.cc:2454] T 8dd1b555d13a4c64b5914f580ea74917 P 9edf436137414f54b26af5a8ec47d81c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8da8827f19c842f6bfae8a9ee27ef69a in term 1.
I20260806 22:49:55.635811 31795 raft_consensus.cc:2454] T 8dd1b555d13a4c64b5914f580ea74917 P 4eaa65e90f624ffaa081592d38dfb33f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8da8827f19c842f6bfae8a9ee27ef69a in term 1.
I20260806 22:49:55.635969 31578 leader_election.cc:302] T 8dd1b555d13a4c64b5914f580ea74917 P 8da8827f19c842f6bfae8a9ee27ef69a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4eaa65e90f624ffaa081592d38dfb33f, 8da8827f19c842f6bfae8a9ee27ef69a; no voters:
I20260806 22:49:55.636106 31872 raft_consensus.cc:2781] T 8dd1b555d13a4c64b5914f580ea74917 P 8da8827f19c842f6bfae8a9ee27ef69a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:49:55.636169 31872 raft_consensus.cc:686] T 8dd1b555d13a4c64b5914f580ea74917 P 8da8827f19c842f6bfae8a9ee27ef69a [term 1 LEADER]: Becoming Leader. State: Replica: 8da8827f19c842f6bfae8a9ee27ef69a, State: Running, Role: LEADER
I20260806 22:49:55.636219 31872 consensus_queue.cc:237] T 8dd1b555d13a4c64b5914f580ea74917 P 8da8827f19c842f6bfae8a9ee27ef69a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9edf436137414f54b26af5a8ec47d81c" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 35231 } } peers { permanent_uuid: "8da8827f19c842f6bfae8a9ee27ef69a" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 33853 } } peers { permanent_uuid: "4eaa65e90f624ffaa081592d38dfb33f" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 41453 } }
I20260806 22:49:55.636456 31877 tablet_replica.cc:439] T 8dd1b555d13a4c64b5914f580ea74917 P 8da8827f19c842f6bfae8a9ee27ef69a: TxnStatusTablet state changed. Reason: New leader 8da8827f19c842f6bfae8a9ee27ef69a. Latest consensus state: current_term: 1 leader_uuid: "8da8827f19c842f6bfae8a9ee27ef69a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9edf436137414f54b26af5a8ec47d81c" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 35231 } } peers { permanent_uuid: "8da8827f19c842f6bfae8a9ee27ef69a" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 33853 } } peers { permanent_uuid: "4eaa65e90f624ffaa081592d38dfb33f" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 41453 } } }
I20260806 22:49:55.636552 31877 tablet_replica.cc:442] T 8dd1b555d13a4c64b5914f580ea74917 P 8da8827f19c842f6bfae8a9ee27ef69a: This TxnStatusTablet replica's current role is: LEADER
I20260806 22:49:55.636709 31889 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 22:49:55.636821 31889 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 22:49:55.636788 31335 catalog_manager.cc:5495] T 8dd1b555d13a4c64b5914f580ea74917 P 8da8827f19c842f6bfae8a9ee27ef69a reported cstate change: term changed from 0 to 1, leader changed from <none> to 8da8827f19c842f6bfae8a9ee27ef69a (127.28.218.194). New cstate: current_term: 1 leader_uuid: "8da8827f19c842f6bfae8a9ee27ef69a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9edf436137414f54b26af5a8ec47d81c" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 35231 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8da8827f19c842f6bfae8a9ee27ef69a" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 33853 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4eaa65e90f624ffaa081592d38dfb33f" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 41453 } health_report { overall_health: UNKNOWN } } }
I20260806 22:49:55.668416 31884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:55.665678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.194:33853 (local address 127.0.0.1:60710)
0806 22:49:55.665751 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:55.665763 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:55.665830 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:55.666052 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:55.666053 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:55.666055 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 22:49:55.666104 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:55.666108 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:55.666784 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:55.666786 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:55.667425 (+ 639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:55.667427 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:55.667852 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:55.667853 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:55.667921 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:55.667934 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 22:49:55.668229 (+ 295us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 22:49:55.668239 (+ 10us) client_negotiation.cc:769] Sending connection context
0806 22:49:55.668297 (+ 58us) client_negotiation.cc:240] Negotiation successful
0806 22:49:55.668347 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260806 22:49:55.668414 31892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:55.665748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60710 (local address 127.28.218.194:33853)
0806 22:49:55.665880 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:55.665882 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:55.665891 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:49:55.665928 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:55.665928 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:55.665946 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:55.666047 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:49:55.666190 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:55.666699 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:55.667541 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:55.667760 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:55.667849 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:55.668062 (+ 213us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:49:55.668160 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:49:55.668250 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:55.668302 (+ 52us) server_negotiation.cc:299] Negotiation successful
0806 22:49:55.668322 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":53,"threads_started":1}
I20260806 22:49:55.670123 31795 raft_consensus.cc:1261] T 8dd1b555d13a4c64b5914f580ea74917 P 4eaa65e90f624ffaa081592d38dfb33f [term 1 FOLLOWER]: Refusing update from remote peer 8da8827f19c842f6bfae8a9ee27ef69a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:49:55.670190 31501 raft_consensus.cc:1261] T 8dd1b555d13a4c64b5914f580ea74917 P 9edf436137414f54b26af5a8ec47d81c [term 1 FOLLOWER]: Refusing update from remote peer 8da8827f19c842f6bfae8a9ee27ef69a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:49:55.670475 31872 consensus_queue.cc:1035] T 8dd1b555d13a4c64b5914f580ea74917 P 8da8827f19c842f6bfae8a9ee27ef69a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9edf436137414f54b26af5a8ec47d81c" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 35231 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:49:55.670639 31877 consensus_queue.cc:1035] T 8dd1b555d13a4c64b5914f580ea74917 P 8da8827f19c842f6bfae8a9ee27ef69a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4eaa65e90f624ffaa081592d38dfb33f" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 41453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:49:55.671828 31875 tablet_replica.cc:439] T 8dd1b555d13a4c64b5914f580ea74917 P 4eaa65e90f624ffaa081592d38dfb33f: TxnStatusTablet state changed. Reason: New leader 8da8827f19c842f6bfae8a9ee27ef69a. Latest consensus state: current_term: 1 leader_uuid: "8da8827f19c842f6bfae8a9ee27ef69a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9edf436137414f54b26af5a8ec47d81c" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 35231 } } peers { permanent_uuid: "8da8827f19c842f6bfae8a9ee27ef69a" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 33853 } } peers { permanent_uuid: "4eaa65e90f624ffaa081592d38dfb33f" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 41453 } } }
I20260806 22:49:55.671931 31875 tablet_replica.cc:442] T 8dd1b555d13a4c64b5914f580ea74917 P 4eaa65e90f624ffaa081592d38dfb33f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:49:55.672000 31874 tablet_replica.cc:439] T 8dd1b555d13a4c64b5914f580ea74917 P 9edf436137414f54b26af5a8ec47d81c: TxnStatusTablet state changed. Reason: New leader 8da8827f19c842f6bfae8a9ee27ef69a. Latest consensus state: current_term: 1 leader_uuid: "8da8827f19c842f6bfae8a9ee27ef69a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9edf436137414f54b26af5a8ec47d81c" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 35231 } } peers { permanent_uuid: "8da8827f19c842f6bfae8a9ee27ef69a" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 33853 } } peers { permanent_uuid: "4eaa65e90f624ffaa081592d38dfb33f" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 41453 } } }
I20260806 22:49:55.672092 31874 tablet_replica.cc:442] T 8dd1b555d13a4c64b5914f580ea74917 P 9edf436137414f54b26af5a8ec47d81c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:49:55.673177 31872 tablet_replica.cc:439] T 8dd1b555d13a4c64b5914f580ea74917 P 8da8827f19c842f6bfae8a9ee27ef69a: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8da8827f19c842f6bfae8a9ee27ef69a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9edf436137414f54b26af5a8ec47d81c" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 35231 } } peers { permanent_uuid: "8da8827f19c842f6bfae8a9ee27ef69a" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 33853 } } peers { permanent_uuid: "4eaa65e90f624ffaa081592d38dfb33f" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 41453 } } }
I20260806 22:49:55.673254 31872 tablet_replica.cc:442] T 8dd1b555d13a4c64b5914f580ea74917 P 8da8827f19c842f6bfae8a9ee27ef69a: This TxnStatusTablet replica's current role is: LEADER
I20260806 22:49:55.673300 31872 tablet_replica.cc:439] T 8dd1b555d13a4c64b5914f580ea74917 P 8da8827f19c842f6bfae8a9ee27ef69a: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8da8827f19c842f6bfae8a9ee27ef69a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9edf436137414f54b26af5a8ec47d81c" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 35231 } } peers { permanent_uuid: "8da8827f19c842f6bfae8a9ee27ef69a" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 33853 } } peers { permanent_uuid: "4eaa65e90f624ffaa081592d38dfb33f" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 41453 } } }
I20260806 22:49:55.673336 31872 tablet_replica.cc:442] T 8dd1b555d13a4c64b5914f580ea74917 P 8da8827f19c842f6bfae8a9ee27ef69a: This TxnStatusTablet replica's current role is: LEADER
I20260806 22:49:55.673384 31893 mvcc.cc:205] Tried to move back new op lower bound from 7315687815862452224 to 7315687815726317568. Current Snapshot: MvccSnapshot[applied={T|T < 7315687815862452224 or (T in {7315687815862452224})}]
I20260806 22:49:55.673379 31875 tablet_replica.cc:439] T 8dd1b555d13a4c64b5914f580ea74917 P 4eaa65e90f624ffaa081592d38dfb33f: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8da8827f19c842f6bfae8a9ee27ef69a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9edf436137414f54b26af5a8ec47d81c" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 35231 } } peers { permanent_uuid: "8da8827f19c842f6bfae8a9ee27ef69a" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 33853 } } peers { permanent_uuid: "4eaa65e90f624ffaa081592d38dfb33f" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 41453 } } }
I20260806 22:49:55.673444 31875 tablet_replica.cc:442] T 8dd1b555d13a4c64b5914f580ea74917 P 4eaa65e90f624ffaa081592d38dfb33f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:49:55.673882 31874 tablet_replica.cc:439] T 8dd1b555d13a4c64b5914f580ea74917 P 9edf436137414f54b26af5a8ec47d81c: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8da8827f19c842f6bfae8a9ee27ef69a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9edf436137414f54b26af5a8ec47d81c" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 35231 } } peers { permanent_uuid: "8da8827f19c842f6bfae8a9ee27ef69a" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 33853 } } peers { permanent_uuid: "4eaa65e90f624ffaa081592d38dfb33f" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 41453 } } }
I20260806 22:49:55.673972 31895 mvcc.cc:205] Tried to move back new op lower bound from 7315687815862452224 to 7315687815726317568. Current Snapshot: MvccSnapshot[applied={T|T < 7315687815862452224}]
I20260806 22:49:55.673975 31874 tablet_replica.cc:442] T 8dd1b555d13a4c64b5914f580ea74917 P 9edf436137414f54b26af5a8ec47d81c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:49:55.674716 31894 mvcc.cc:205] Tried to move back new op lower bound from 7315687815862452224 to 7315687815726317568. Current Snapshot: MvccSnapshot[applied={T|T < 7315687815862452224}]
I20260806 22:49:55.677970 31892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:55.675094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60724 (local address 127.28.218.194:33853)
0806 22:49:55.675167 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:55.675169 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:55.675647 (+ 478us) server_negotiation.cc:407] Connection header received
0806 22:49:55.675671 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:55.675672 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:55.675682 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:55.675732 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:49:55.675810 (+ 78us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:55.676547 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:55.677268 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:55.677566 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:55.677635 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:55.677754 (+ 119us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:49:55.677786 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:49:55.677836 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:55.677882 (+ 46us) server_negotiation.cc:299] Negotiation successful
0806 22:49:55.677900 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 22:49:55.682930 31892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:55.680215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.194:59105 (local address 127.28.218.194:33853)
0806 22:49:55.680267 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:55.680269 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:55.680375 (+ 106us) server_negotiation.cc:407] Connection header received
0806 22:49:55.680449 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:55.680450 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:55.680471 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:55.680525 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:49:55.680690 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:55.681094 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:55.682221 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:55.682470 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:55.682521 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:55.682741 (+ 220us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:49:55.682761 (+ 20us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:49:55.682802 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:55.682876 (+ 74us) server_negotiation.cc:299] Negotiation successful
0806 22:49:55.682889 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 22:49:55.683105 31906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:55.680145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.194:33853 (local address 127.28.218.194:59105)
0806 22:49:55.680314 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:55.680324 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:55.680381 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:55.680541 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:55.680542 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:55.680543 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 22:49:55.680619 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:55.680620 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:55.681190 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:55.681191 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:55.682145 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:55.682148 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:55.682554 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:55.682555 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:55.682638 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:55.682651 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 22:49:55.682824 (+ 173us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 22:49:55.682826 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 22:49:55.682875 (+ 49us) client_negotiation.cc:240] Negotiation successful
0806 22:49:55.682923 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":110,"threads_started":1}
I20260806 22:49:55.685192 31795 raft_consensus.cc:1261] T 19a3c5d01ab344db8b6ca7a2faf14e45 P 4eaa65e90f624ffaa081592d38dfb33f [term 1 FOLLOWER]: Refusing update from remote peer 8da8827f19c842f6bfae8a9ee27ef69a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:49:55.685204 31501 raft_consensus.cc:1261] T 19a3c5d01ab344db8b6ca7a2faf14e45 P 9edf436137414f54b26af5a8ec47d81c [term 1 FOLLOWER]: Refusing update from remote peer 8da8827f19c842f6bfae8a9ee27ef69a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:49:55.685395 31898 consensus_queue.cc:1035] T 19a3c5d01ab344db8b6ca7a2faf14e45 P 8da8827f19c842f6bfae8a9ee27ef69a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4eaa65e90f624ffaa081592d38dfb33f" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 41453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:49:55.685468 31872 consensus_queue.cc:1035] T 19a3c5d01ab344db8b6ca7a2faf14e45 P 8da8827f19c842f6bfae8a9ee27ef69a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9edf436137414f54b26af5a8ec47d81c" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 35231 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:49:55.697057 31333 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32916:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:49:55.697165 31333 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32916:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:49:55.697837 31333 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d5a2faa728f142b79312f4cb47b13e55: Sending DeleteTablet for 3 replicas of tablet 19a3c5d01ab344db8b6ca7a2faf14e45
I20260806 22:49:55.698155 31478 tablet_service.cc:1514] Processing DeleteTablet for tablet 19a3c5d01ab344db8b6ca7a2faf14e45 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:49:55 UTC) from {username='kudu', principal='kudu/127.28.218.254@KRBTEST.COM'} at 127.0.0.1:41908
I20260806 22:49:55.698161 31774 tablet_service.cc:1514] Processing DeleteTablet for tablet 19a3c5d01ab344db8b6ca7a2faf14e45 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:49:55 UTC) from {username='kudu', principal='kudu/127.28.218.254@KRBTEST.COM'} at 127.0.0.1:60258
I20260806 22:49:55.698359 31627 tablet_service.cc:1514] Processing DeleteTablet for tablet 19a3c5d01ab344db8b6ca7a2faf14e45 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:49:55 UTC) from {username='kudu', principal='kudu/127.28.218.254@KRBTEST.COM'} at 127.0.0.1:60708
I20260806 22:49:55.698611 31912 tablet_replica.cc:330] T 19a3c5d01ab344db8b6ca7a2faf14e45 P 4eaa65e90f624ffaa081592d38dfb33f: stopping tablet replica
I20260806 22:49:55.698614 31333 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32916:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 22:49:55.698719 31912 raft_consensus.cc:2229] T 19a3c5d01ab344db8b6ca7a2faf14e45 P 4eaa65e90f624ffaa081592d38dfb33f [term 1 FOLLOWER]: Raft consensus shutting down.
W20260806 22:49:55.698743 31333 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:49:55.698770 31912 raft_consensus.cc:2258] T 19a3c5d01ab344db8b6ca7a2faf14e45 P 4eaa65e90f624ffaa081592d38dfb33f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:49:55.699111 31910 tablet_replica.cc:330] T 19a3c5d01ab344db8b6ca7a2faf14e45 P 9edf436137414f54b26af5a8ec47d81c: stopping tablet replica
I20260806 22:49:55.699270 31910 raft_consensus.cc:2229] T 19a3c5d01ab344db8b6ca7a2faf14e45 P 9edf436137414f54b26af5a8ec47d81c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:49:55.699331 31910 raft_consensus.cc:2258] T 19a3c5d01ab344db8b6ca7a2faf14e45 P 9edf436137414f54b26af5a8ec47d81c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:49:55.699681 31912 ts_tablet_manager.cc:1905] T 19a3c5d01ab344db8b6ca7a2faf14e45 P 4eaa65e90f624ffaa081592d38dfb33f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:49:55.699858 31910 ts_tablet_manager.cc:1905] T 19a3c5d01ab344db8b6ca7a2faf14e45 P 9edf436137414f54b26af5a8ec47d81c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:49:55.700325 31478 tablet_service.cc:1467] Processing CreateTablet for tablet 3fb4e815adb84a509eb38ce8bdf8fc5d (DEFAULT_TABLE table=test-table [id=298c0d1972c24689a41dd7ca257363c0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:49:55.700464 31478 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3fb4e815adb84a509eb38ce8bdf8fc5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:49:55.701538 31912 ts_tablet_manager.cc:1918] T 19a3c5d01ab344db8b6ca7a2faf14e45 P 4eaa65e90f624ffaa081592d38dfb33f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 22:49:55.701609 31912 log.cc:1196] T 19a3c5d01ab344db8b6ca7a2faf14e45 P 4eaa65e90f624ffaa081592d38dfb33f: Deleting WAL directory at /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056586788339-29547-0/minicluster-data/ts-2/wal/wals/19a3c5d01ab344db8b6ca7a2faf14e45
I20260806 22:49:55.701889 31870 tablet_bootstrap.cc:492] T 3fb4e815adb84a509eb38ce8bdf8fc5d P 9edf436137414f54b26af5a8ec47d81c: Bootstrap starting.
I20260806 22:49:55.701943 31627 tablet_service.cc:1467] Processing CreateTablet for tablet 3fb4e815adb84a509eb38ce8bdf8fc5d (DEFAULT_TABLE table=test-table [id=298c0d1972c24689a41dd7ca257363c0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:49:55.702072 31774 tablet_service.cc:1467] Processing CreateTablet for tablet 3fb4e815adb84a509eb38ce8bdf8fc5d (DEFAULT_TABLE table=test-table [id=298c0d1972c24689a41dd7ca257363c0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:49:55.702219 31774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3fb4e815adb84a509eb38ce8bdf8fc5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:49:55.702344 31870 tablet_bootstrap.cc:654] T 3fb4e815adb84a509eb38ce8bdf8fc5d P 9edf436137414f54b26af5a8ec47d81c: Neither blocks nor log segments found. Creating new log.
I20260806 22:49:55.702534 31910 ts_tablet_manager.cc:1918] T 19a3c5d01ab344db8b6ca7a2faf14e45 P 9edf436137414f54b26af5a8ec47d81c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 22:49:55.702667 31910 log.cc:1196] T 19a3c5d01ab344db8b6ca7a2faf14e45 P 9edf436137414f54b26af5a8ec47d81c: Deleting WAL directory at /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056586788339-29547-0/minicluster-data/ts-0/wal/wals/19a3c5d01ab344db8b6ca7a2faf14e45
I20260806 22:49:55.702688 31912 ts_tablet_manager.cc:1939] T 19a3c5d01ab344db8b6ca7a2faf14e45 P 4eaa65e90f624ffaa081592d38dfb33f: Deleting consensus metadata
I20260806 22:49:55.703023 31910 ts_tablet_manager.cc:1939] T 19a3c5d01ab344db8b6ca7a2faf14e45 P 9edf436137414f54b26af5a8ec47d81c: Deleting consensus metadata
I20260806 22:49:55.703090 31322 catalog_manager.cc:4841] TS 4eaa65e90f624ffaa081592d38dfb33f (127.28.218.195:41453): tablet 19a3c5d01ab344db8b6ca7a2faf14e45 (table test-table [id=eecf0e05786942198649dc4b96bf651d]) successfully deleted
I20260806 22:49:55.703529 31870 tablet_bootstrap.cc:492] T 3fb4e815adb84a509eb38ce8bdf8fc5d P 9edf436137414f54b26af5a8ec47d81c: No bootstrap required, opened a new log
I20260806 22:49:55.703584 31870 ts_tablet_manager.cc:1397] T 3fb4e815adb84a509eb38ce8bdf8fc5d P 9edf436137414f54b26af5a8ec47d81c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:49:55.703704 31868 tablet_bootstrap.cc:492] T 3fb4e815adb84a509eb38ce8bdf8fc5d P 4eaa65e90f624ffaa081592d38dfb33f: Bootstrap starting.
I20260806 22:49:55.703724 31870 raft_consensus.cc:348] T 3fb4e815adb84a509eb38ce8bdf8fc5d P 9edf436137414f54b26af5a8ec47d81c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9edf436137414f54b26af5a8ec47d81c" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 35231 } } peers { permanent_uuid: "8da8827f19c842f6bfae8a9ee27ef69a" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 33853 } } peers { permanent_uuid: "4eaa65e90f624ffaa081592d38dfb33f" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 41453 } }
I20260806 22:49:55.703792 31870 raft_consensus.cc:374] T 3fb4e815adb84a509eb38ce8bdf8fc5d P 9edf436137414f54b26af5a8ec47d81c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:49:55.703843 31870 raft_consensus.cc:729] T 3fb4e815adb84a509eb38ce8bdf8fc5d P 9edf436137414f54b26af5a8ec47d81c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9edf436137414f54b26af5a8ec47d81c, State: Initialized, Role: FOLLOWER
I20260806 22:49:55.703893 31870 consensus_queue.cc:260] T 3fb4e815adb84a509eb38ce8bdf8fc5d P 9edf436137414f54b26af5a8ec47d81c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9edf436137414f54b26af5a8ec47d81c" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 35231 } } peers { permanent_uuid: "8da8827f19c842f6bfae8a9ee27ef69a" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 33853 } } peers { permanent_uuid: "4eaa65e90f624ffaa081592d38dfb33f" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 41453 } }
I20260806 22:49:55.703997 31870 ts_tablet_manager.cc:1428] T 3fb4e815adb84a509eb38ce8bdf8fc5d P 9edf436137414f54b26af5a8ec47d81c: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260806 22:49:55.704133 31868 tablet_bootstrap.cc:654] T 3fb4e815adb84a509eb38ce8bdf8fc5d P 4eaa65e90f624ffaa081592d38dfb33f: Neither blocks nor log segments found. Creating new log.
I20260806 22:49:55.704766 31320 catalog_manager.cc:4841] TS 9edf436137414f54b26af5a8ec47d81c (127.28.218.193:35231): tablet 19a3c5d01ab344db8b6ca7a2faf14e45 (table test-table [id=eecf0e05786942198649dc4b96bf651d]) successfully deleted
I20260806 22:49:55.705098 31868 tablet_bootstrap.cc:492] T 3fb4e815adb84a509eb38ce8bdf8fc5d P 4eaa65e90f624ffaa081592d38dfb33f: No bootstrap required, opened a new log
I20260806 22:49:55.705140 31868 ts_tablet_manager.cc:1397] T 3fb4e815adb84a509eb38ce8bdf8fc5d P 4eaa65e90f624ffaa081592d38dfb33f: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 22:49:55.705281 31868 raft_consensus.cc:348] T 3fb4e815adb84a509eb38ce8bdf8fc5d P 4eaa65e90f624ffaa081592d38dfb33f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9edf436137414f54b26af5a8ec47d81c" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 35231 } } peers { permanent_uuid: "8da8827f19c842f6bfae8a9ee27ef69a" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 33853 } } peers { permanent_uuid: "4eaa65e90f624ffaa081592d38dfb33f" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 41453 } }
I20260806 22:49:55.705343 31868 raft_consensus.cc:374] T 3fb4e815adb84a509eb38ce8bdf8fc5d P 4eaa65e90f624ffaa081592d38dfb33f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:49:55.705358 31868 raft_consensus.cc:729] T 3fb4e815adb84a509eb38ce8bdf8fc5d P 4eaa65e90f624ffaa081592d38dfb33f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4eaa65e90f624ffaa081592d38dfb33f, State: Initialized, Role: FOLLOWER
I20260806 22:49:55.705425 31868 consensus_queue.cc:260] T 3fb4e815adb84a509eb38ce8bdf8fc5d P 4eaa65e90f624ffaa081592d38dfb33f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9edf436137414f54b26af5a8ec47d81c" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 35231 } } peers { permanent_uuid: "8da8827f19c842f6bfae8a9ee27ef69a" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 33853 } } peers { permanent_uuid: "4eaa65e90f624ffaa081592d38dfb33f" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 41453 } }
I20260806 22:49:55.705523 31868 ts_tablet_manager.cc:1428] T 3fb4e815adb84a509eb38ce8bdf8fc5d P 4eaa65e90f624ffaa081592d38dfb33f: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260806 22:49:55.705837 31627 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3fb4e815adb84a509eb38ce8bdf8fc5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:49:55.706821 31866 tablet_bootstrap.cc:492] T 3fb4e815adb84a509eb38ce8bdf8fc5d P 8da8827f19c842f6bfae8a9ee27ef69a: Bootstrap starting.
I20260806 22:49:55.707086 31913 tablet_replica.cc:330] T 19a3c5d01ab344db8b6ca7a2faf14e45 P 8da8827f19c842f6bfae8a9ee27ef69a: stopping tablet replica
I20260806 22:49:55.707158 31866 tablet_bootstrap.cc:654] T 3fb4e815adb84a509eb38ce8bdf8fc5d P 8da8827f19c842f6bfae8a9ee27ef69a: Neither blocks nor log segments found. Creating new log.
I20260806 22:49:55.707207 31913 raft_consensus.cc:2229] T 19a3c5d01ab344db8b6ca7a2faf14e45 P 8da8827f19c842f6bfae8a9ee27ef69a [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:49:55.707355 31913 raft_consensus.cc:2258] T 19a3c5d01ab344db8b6ca7a2faf14e45 P 8da8827f19c842f6bfae8a9ee27ef69a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:49:55.707726 31913 ts_tablet_manager.cc:1905] T 19a3c5d01ab344db8b6ca7a2faf14e45 P 8da8827f19c842f6bfae8a9ee27ef69a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:49:55.707823 31866 tablet_bootstrap.cc:492] T 3fb4e815adb84a509eb38ce8bdf8fc5d P 8da8827f19c842f6bfae8a9ee27ef69a: No bootstrap required, opened a new log
I20260806 22:49:55.707865 31866 ts_tablet_manager.cc:1397] T 3fb4e815adb84a509eb38ce8bdf8fc5d P 8da8827f19c842f6bfae8a9ee27ef69a: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260806 22:49:55.708019 31866 raft_consensus.cc:348] T 3fb4e815adb84a509eb38ce8bdf8fc5d P 8da8827f19c842f6bfae8a9ee27ef69a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9edf436137414f54b26af5a8ec47d81c" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 35231 } } peers { permanent_uuid: "8da8827f19c842f6bfae8a9ee27ef69a" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 33853 } } peers { permanent_uuid: "4eaa65e90f624ffaa081592d38dfb33f" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 41453 } }
I20260806 22:49:55.708089 31866 raft_consensus.cc:374] T 3fb4e815adb84a509eb38ce8bdf8fc5d P 8da8827f19c842f6bfae8a9ee27ef69a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:49:55.708112 31866 raft_consensus.cc:729] T 3fb4e815adb84a509eb38ce8bdf8fc5d P 8da8827f19c842f6bfae8a9ee27ef69a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8da8827f19c842f6bfae8a9ee27ef69a, State: Initialized, Role: FOLLOWER
I20260806 22:49:55.708156 31866 consensus_queue.cc:260] T 3fb4e815adb84a509eb38ce8bdf8fc5d P 8da8827f19c842f6bfae8a9ee27ef69a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9edf436137414f54b26af5a8ec47d81c" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 35231 } } peers { permanent_uuid: "8da8827f19c842f6bfae8a9ee27ef69a" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 33853 } } peers { permanent_uuid: "4eaa65e90f624ffaa081592d38dfb33f" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 41453 } }
I20260806 22:49:55.708256 31866 ts_tablet_manager.cc:1428] T 3fb4e815adb84a509eb38ce8bdf8fc5d P 8da8827f19c842f6bfae8a9ee27ef69a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:55.709532 31913 ts_tablet_manager.cc:1918] T 19a3c5d01ab344db8b6ca7a2faf14e45 P 8da8827f19c842f6bfae8a9ee27ef69a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 22:49:55.709589 31913 log.cc:1196] T 19a3c5d01ab344db8b6ca7a2faf14e45 P 8da8827f19c842f6bfae8a9ee27ef69a: Deleting WAL directory at /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056586788339-29547-0/minicluster-data/ts-1/wal/wals/19a3c5d01ab344db8b6ca7a2faf14e45
I20260806 22:49:55.709831 31913 ts_tablet_manager.cc:1939] T 19a3c5d01ab344db8b6ca7a2faf14e45 P 8da8827f19c842f6bfae8a9ee27ef69a: Deleting consensus metadata
I20260806 22:49:55.710227 31321 catalog_manager.cc:4841] TS 8da8827f19c842f6bfae8a9ee27ef69a (127.28.218.194:33853): tablet 19a3c5d01ab344db8b6ca7a2faf14e45 (table test-table [id=eecf0e05786942198649dc4b96bf651d]) successfully deleted
I20260806 22:49:55.823515 31874 raft_consensus.cc:482] T 3fb4e815adb84a509eb38ce8bdf8fc5d P 9edf436137414f54b26af5a8ec47d81c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:49:55.823633 31874 raft_consensus.cc:504] T 3fb4e815adb84a509eb38ce8bdf8fc5d P 9edf436137414f54b26af5a8ec47d81c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9edf436137414f54b26af5a8ec47d81c" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 35231 } } peers { permanent_uuid: "8da8827f19c842f6bfae8a9ee27ef69a" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 33853 } } peers { permanent_uuid: "4eaa65e90f624ffaa081592d38dfb33f" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 41453 } }
I20260806 22:49:55.824020 31874 leader_election.cc:288] T 3fb4e815adb84a509eb38ce8bdf8fc5d P 9edf436137414f54b26af5a8ec47d81c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8da8827f19c842f6bfae8a9ee27ef69a (127.28.218.194:33853), 4eaa65e90f624ffaa081592d38dfb33f (127.28.218.195:41453)
I20260806 22:49:55.827621 31861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:55.824373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.193:47589 (local address 127.28.218.195:41453)
0806 22:49:55.824422 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:55.824424 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:55.824542 (+ 118us) server_negotiation.cc:407] Connection header received
0806 22:49:55.824685 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:55.824686 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:55.824695 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:55.824729 (+ 34us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:49:55.824945 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:55.825637 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:55.826845 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:55.827263 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:55.827342 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:55.827351 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:55.827513 (+ 162us) server_negotiation.cc:299] Negotiation successful
0806 22:49:55.827560 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 22:49:55.827634 31917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:55.824195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.195:41453 (local address 127.28.218.193:47589)
0806 22:49:55.824446 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:55.824457 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:55.824572 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:55.824802 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:55.824803 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:55.824805 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:49:55.824863 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:55.824864 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:55.825753 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:55.825754 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:55.826762 (+ 1008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:55.826764 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:55.827352 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:55.827353 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:55.827421 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:55.827425 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:49:55.827501 (+ 76us) client_negotiation.cc:240] Negotiation successful
0806 22:49:55.827542 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":233,"mutex_wait_us":3,"thread_start_us":107,"threads_started":1}
I20260806 22:49:55.827702 31892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:55.824304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.193:52379 (local address 127.28.218.194:33853)
0806 22:49:55.824348 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:55.824350 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:55.824535 (+ 185us) server_negotiation.cc:407] Connection header received
0806 22:49:55.824572 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:55.824573 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:55.824583 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:55.824636 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:49:55.824819 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:55.825328 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:55.826713 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:55.827309 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:55.827384 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:55.827393 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:55.827592 (+ 199us) server_negotiation.cc:299] Negotiation successful
0806 22:49:55.827634 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 22:49:55.827950 31918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:55.824195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.194:33853 (local address 127.28.218.193:52379)
0806 22:49:55.824434 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:55.824446 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:55.824520 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:55.824673 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:55.824674 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:55.824675 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:49:55.824741 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:55.824743 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:55.825411 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:55.825412 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:55.826604 (+ 1192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:55.826606 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:55.827431 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:55.827432 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:55.827500 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:55.827504 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:49:55.827815 (+ 311us) client_negotiation.cc:240] Negotiation successful
0806 22:49:55.827858 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":108,"threads_started":1}
I20260806 22:49:55.827998 31795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3fb4e815adb84a509eb38ce8bdf8fc5d" candidate_uuid: "9edf436137414f54b26af5a8ec47d81c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4eaa65e90f624ffaa081592d38dfb33f" is_pre_election: true
I20260806 22:49:55.828081 31795 raft_consensus.cc:2454] T 3fb4e815adb84a509eb38ce8bdf8fc5d P 4eaa65e90f624ffaa081592d38dfb33f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9edf436137414f54b26af5a8ec47d81c in term 0.
I20260806 22:49:55.828279 31431 leader_election.cc:302] T 3fb4e815adb84a509eb38ce8bdf8fc5d P 9edf436137414f54b26af5a8ec47d81c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4eaa65e90f624ffaa081592d38dfb33f, 9edf436137414f54b26af5a8ec47d81c; no voters:
I20260806 22:49:55.828240 31649 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3fb4e815adb84a509eb38ce8bdf8fc5d" candidate_uuid: "9edf436137414f54b26af5a8ec47d81c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8da8827f19c842f6bfae8a9ee27ef69a" is_pre_election: true
I20260806 22:49:55.828380 31649 raft_consensus.cc:2454] T 3fb4e815adb84a509eb38ce8bdf8fc5d P 8da8827f19c842f6bfae8a9ee27ef69a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9edf436137414f54b26af5a8ec47d81c in term 0.
I20260806 22:49:55.828404 31874 raft_consensus.cc:2781] T 3fb4e815adb84a509eb38ce8bdf8fc5d P 9edf436137414f54b26af5a8ec47d81c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:49:55.828437 31874 raft_consensus.cc:482] T 3fb4e815adb84a509eb38ce8bdf8fc5d P 9edf436137414f54b26af5a8ec47d81c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:49:55.828464 31874 raft_consensus.cc:3037] T 3fb4e815adb84a509eb38ce8bdf8fc5d P 9edf436137414f54b26af5a8ec47d81c [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:49:55.829419 31874 raft_consensus.cc:504] T 3fb4e815adb84a509eb38ce8bdf8fc5d P 9edf436137414f54b26af5a8ec47d81c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9edf436137414f54b26af5a8ec47d81c" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 35231 } } peers { permanent_uuid: "8da8827f19c842f6bfae8a9ee27ef69a" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 33853 } } peers { permanent_uuid: "4eaa65e90f624ffaa081592d38dfb33f" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 41453 } }
I20260806 22:49:55.829582 31874 leader_election.cc:288] T 3fb4e815adb84a509eb38ce8bdf8fc5d P 9edf436137414f54b26af5a8ec47d81c [CANDIDATE]: Term 1 election: Requested vote from peers 8da8827f19c842f6bfae8a9ee27ef69a (127.28.218.194:33853), 4eaa65e90f624ffaa081592d38dfb33f (127.28.218.195:41453)
I20260806 22:49:55.829708 31649 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3fb4e815adb84a509eb38ce8bdf8fc5d" candidate_uuid: "9edf436137414f54b26af5a8ec47d81c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8da8827f19c842f6bfae8a9ee27ef69a"
I20260806 22:49:55.829707 31795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3fb4e815adb84a509eb38ce8bdf8fc5d" candidate_uuid: "9edf436137414f54b26af5a8ec47d81c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4eaa65e90f624ffaa081592d38dfb33f"
I20260806 22:49:55.829773 31795 raft_consensus.cc:3037] T 3fb4e815adb84a509eb38ce8bdf8fc5d P 4eaa65e90f624ffaa081592d38dfb33f [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:49:55.829772 31649 raft_consensus.cc:3037] T 3fb4e815adb84a509eb38ce8bdf8fc5d P 8da8827f19c842f6bfae8a9ee27ef69a [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:49:55.830516 31795 raft_consensus.cc:2454] T 3fb4e815adb84a509eb38ce8bdf8fc5d P 4eaa65e90f624ffaa081592d38dfb33f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9edf436137414f54b26af5a8ec47d81c in term 1.
I20260806 22:49:55.830531 31649 raft_consensus.cc:2454] T 3fb4e815adb84a509eb38ce8bdf8fc5d P 8da8827f19c842f6bfae8a9ee27ef69a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9edf436137414f54b26af5a8ec47d81c in term 1.
I20260806 22:49:55.831038 31430 leader_election.cc:302] T 3fb4e815adb84a509eb38ce8bdf8fc5d P 9edf436137414f54b26af5a8ec47d81c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8da8827f19c842f6bfae8a9ee27ef69a, 9edf436137414f54b26af5a8ec47d81c; no voters:
I20260806 22:49:55.831172 31874 raft_consensus.cc:2781] T 3fb4e815adb84a509eb38ce8bdf8fc5d P 9edf436137414f54b26af5a8ec47d81c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:49:55.831310 31874 raft_consensus.cc:686] T 3fb4e815adb84a509eb38ce8bdf8fc5d P 9edf436137414f54b26af5a8ec47d81c [term 1 LEADER]: Becoming Leader. State: Replica: 9edf436137414f54b26af5a8ec47d81c, State: Running, Role: LEADER
I20260806 22:49:55.831382 31874 consensus_queue.cc:237] T 3fb4e815adb84a509eb38ce8bdf8fc5d P 9edf436137414f54b26af5a8ec47d81c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9edf436137414f54b26af5a8ec47d81c" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 35231 } } peers { permanent_uuid: "8da8827f19c842f6bfae8a9ee27ef69a" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 33853 } } peers { permanent_uuid: "4eaa65e90f624ffaa081592d38dfb33f" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 41453 } }
I20260806 22:49:55.832116 31333 catalog_manager.cc:5495] T 3fb4e815adb84a509eb38ce8bdf8fc5d P 9edf436137414f54b26af5a8ec47d81c reported cstate change: term changed from 0 to 1, leader changed from <none> to 9edf436137414f54b26af5a8ec47d81c (127.28.218.193). New cstate: current_term: 1 leader_uuid: "9edf436137414f54b26af5a8ec47d81c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9edf436137414f54b26af5a8ec47d81c" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 35231 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8da8827f19c842f6bfae8a9ee27ef69a" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 33853 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4eaa65e90f624ffaa081592d38dfb33f" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 41453 } health_report { overall_health: UNKNOWN } } }
I20260806 22:49:55.857326 31862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:55.854983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41918 (local address 127.28.218.193:35231)
0806 22:49:55.855044 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:55.855046 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:55.855054 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:49:55.855091 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:55.855092 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:55.855110 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:55.855208 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:49:55.855310 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:55.855775 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:55.856531 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:55.856734 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:55.856804 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:55.856971 (+ 167us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:49:55.857080 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:49:55.857179 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:55.857233 (+ 54us) server_negotiation.cc:299] Negotiation successful
0806 22:49:55.857251 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 22:49:55.862156 31892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:55.859834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.193:56731 (local address 127.28.218.194:33853)
0806 22:49:55.859884 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:55.859886 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:55.859900 (+ 14us) server_negotiation.cc:407] Connection header received
0806 22:49:55.859918 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:55.859919 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:55.859928 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:55.859982 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:49:55.860105 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:55.860514 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:55.861529 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:55.861751 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:55.861796 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:55.861992 (+ 196us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:49:55.862015 (+ 23us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:49:55.862051 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:55.862091 (+ 40us) server_negotiation.cc:299] Negotiation successful
0806 22:49:55.862109 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 22:49:55.862218 31923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:55.859665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.194:33853 (local address 127.28.218.193:56731)
0806 22:49:55.859812 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:55.859821 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:55.859862 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:55.859989 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:55.859989 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:55.859990 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 22:49:55.860052 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:55.860053 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:55.860586 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:55.860587 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:55.861447 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:55.861449 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:55.861831 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:55.861832 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:55.861892 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:55.861906 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 22:49:55.862051 (+ 145us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 22:49:55.862052 (+ 1us) client_negotiation.cc:769] Sending connection context
0806 22:49:55.862090 (+ 38us) client_negotiation.cc:240] Negotiation successful
0806 22:49:55.862127 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":74,"threads_started":1}
I20260806 22:49:55.864343 31795 raft_consensus.cc:1261] T 3fb4e815adb84a509eb38ce8bdf8fc5d P 4eaa65e90f624ffaa081592d38dfb33f [term 1 FOLLOWER]: Refusing update from remote peer 9edf436137414f54b26af5a8ec47d81c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:49:55.864495 31649 raft_consensus.cc:1261] T 3fb4e815adb84a509eb38ce8bdf8fc5d P 8da8827f19c842f6bfae8a9ee27ef69a [term 1 FOLLOWER]: Refusing update from remote peer 9edf436137414f54b26af5a8ec47d81c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:49:55.864593 31919 consensus_queue.cc:1035] T 3fb4e815adb84a509eb38ce8bdf8fc5d P 9edf436137414f54b26af5a8ec47d81c [LEADER]: Connected to new peer: Peer: permanent_uuid: "4eaa65e90f624ffaa081592d38dfb33f" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 41453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:49:55.864751 31919 consensus_queue.cc:1035] T 3fb4e815adb84a509eb38ce8bdf8fc5d P 9edf436137414f54b26af5a8ec47d81c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8da8827f19c842f6bfae8a9ee27ef69a" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 33853 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:49:55.872521 31862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:55.870307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.194:43427 (local address 127.28.218.193:35231)
0806 22:49:55.870341 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:55.870343 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:55.870349 (+ 6us) server_negotiation.cc:407] Connection header received
0806 22:49:55.870367 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:55.870368 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:55.870377 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:55.870421 (+ 44us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:49:55.870568 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:55.871012 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:55.871802 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:55.872059 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:55.872115 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:55.872321 (+ 206us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:49:55.872349 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:49:55.872397 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:55.872442 (+ 45us) server_negotiation.cc:299] Negotiation successful
0806 22:49:55.872459 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 22:49:55.872536 31906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:55.870145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.193:35231 (local address 127.28.218.194:43427)
0806 22:49:55.870184 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:55.870195 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:55.870236 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:55.870459 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:55.870459 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:55.870461 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 22:49:55.870498 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:55.870499 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:55.871094 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:55.871095 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:55.871671 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:55.871673 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:55.872164 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:55.872165 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:55.872219 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:55.872225 (+ 6us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 22:49:55.872400 (+ 175us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 22:49:55.872401 (+ 1us) client_negotiation.cc:769] Sending connection context
0806 22:49:55.872442 (+ 41us) client_negotiation.cc:240] Negotiation successful
0806 22:49:55.872479 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20}
W20260806 22:49:55.872975 31895 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260806 22:49:57.394529 31334 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32916:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:49:57.394626 31334 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32916:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:49:57.395538 31334 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d5a2faa728f142b79312f4cb47b13e55: Sending DeleteTablet for 3 replicas of tablet 3fb4e815adb84a509eb38ce8bdf8fc5d
I20260806 22:49:57.395789 31626 tablet_service.cc:1514] Processing DeleteTablet for tablet 3fb4e815adb84a509eb38ce8bdf8fc5d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:49:57 UTC) from {username='kudu', principal='kudu/127.28.218.254@KRBTEST.COM'} at 127.0.0.1:60708
I20260806 22:49:57.395807 31478 tablet_service.cc:1514] Processing DeleteTablet for tablet 3fb4e815adb84a509eb38ce8bdf8fc5d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:49:57 UTC) from {username='kudu', principal='kudu/127.28.218.254@KRBTEST.COM'} at 127.0.0.1:41908
I20260806 22:49:57.396025 31952 tablet_replica.cc:330] T 3fb4e815adb84a509eb38ce8bdf8fc5d P 9edf436137414f54b26af5a8ec47d81c: stopping tablet replica
I20260806 22:49:57.396087 31774 tablet_service.cc:1514] Processing DeleteTablet for tablet 3fb4e815adb84a509eb38ce8bdf8fc5d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:49:57 UTC) from {username='kudu', principal='kudu/127.28.218.254@KRBTEST.COM'} at 127.0.0.1:60258
I20260806 22:49:57.396111 31952 raft_consensus.cc:2229] T 3fb4e815adb84a509eb38ce8bdf8fc5d P 9edf436137414f54b26af5a8ec47d81c [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:49:57.396211 31952 raft_consensus.cc:2258] T 3fb4e815adb84a509eb38ce8bdf8fc5d P 9edf436137414f54b26af5a8ec47d81c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:49:57.396261 31953 tablet_replica.cc:330] T 3fb4e815adb84a509eb38ce8bdf8fc5d P 4eaa65e90f624ffaa081592d38dfb33f: stopping tablet replica
I20260806 22:49:57.396315 31951 tablet_replica.cc:330] T 3fb4e815adb84a509eb38ce8bdf8fc5d P 8da8827f19c842f6bfae8a9ee27ef69a: stopping tablet replica
I20260806 22:49:57.396327 31953 raft_consensus.cc:2229] T 3fb4e815adb84a509eb38ce8bdf8fc5d P 4eaa65e90f624ffaa081592d38dfb33f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:49:57.396368 31953 raft_consensus.cc:2258] T 3fb4e815adb84a509eb38ce8bdf8fc5d P 4eaa65e90f624ffaa081592d38dfb33f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:49:57.396378 31951 raft_consensus.cc:2229] T 3fb4e815adb84a509eb38ce8bdf8fc5d P 8da8827f19c842f6bfae8a9ee27ef69a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:49:57.396423 31951 raft_consensus.cc:2258] T 3fb4e815adb84a509eb38ce8bdf8fc5d P 8da8827f19c842f6bfae8a9ee27ef69a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:49:57.396662 31951 ts_tablet_manager.cc:1905] T 3fb4e815adb84a509eb38ce8bdf8fc5d P 8da8827f19c842f6bfae8a9ee27ef69a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:49:57.396698 31953 ts_tablet_manager.cc:1905] T 3fb4e815adb84a509eb38ce8bdf8fc5d P 4eaa65e90f624ffaa081592d38dfb33f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:49:57.398211 31953 ts_tablet_manager.cc:1918] T 3fb4e815adb84a509eb38ce8bdf8fc5d P 4eaa65e90f624ffaa081592d38dfb33f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 22:49:57.398473 31952 ts_tablet_manager.cc:1905] T 3fb4e815adb84a509eb38ce8bdf8fc5d P 9edf436137414f54b26af5a8ec47d81c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:49:57.398599 31953 log.cc:1196] T 3fb4e815adb84a509eb38ce8bdf8fc5d P 4eaa65e90f624ffaa081592d38dfb33f: Deleting WAL directory at /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056586788339-29547-0/minicluster-data/ts-2/wal/wals/3fb4e815adb84a509eb38ce8bdf8fc5d
I20260806 22:49:57.398638 31951 ts_tablet_manager.cc:1918] T 3fb4e815adb84a509eb38ce8bdf8fc5d P 8da8827f19c842f6bfae8a9ee27ef69a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 22:49:57.398748 31951 log.cc:1196] T 3fb4e815adb84a509eb38ce8bdf8fc5d P 8da8827f19c842f6bfae8a9ee27ef69a: Deleting WAL directory at /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056586788339-29547-0/minicluster-data/ts-1/wal/wals/3fb4e815adb84a509eb38ce8bdf8fc5d
I20260806 22:49:57.399039 31953 ts_tablet_manager.cc:1939] T 3fb4e815adb84a509eb38ce8bdf8fc5d P 4eaa65e90f624ffaa081592d38dfb33f: Deleting consensus metadata
I20260806 22:49:57.399039 31951 ts_tablet_manager.cc:1939] T 3fb4e815adb84a509eb38ce8bdf8fc5d P 8da8827f19c842f6bfae8a9ee27ef69a: Deleting consensus metadata
I20260806 22:49:57.399430 31321 catalog_manager.cc:4841] TS 8da8827f19c842f6bfae8a9ee27ef69a (127.28.218.194:33853): tablet 3fb4e815adb84a509eb38ce8bdf8fc5d (table test-table [id=298c0d1972c24689a41dd7ca257363c0]) successfully deleted
I20260806 22:49:57.399475 31322 catalog_manager.cc:4841] TS 4eaa65e90f624ffaa081592d38dfb33f (127.28.218.195:41453): tablet 3fb4e815adb84a509eb38ce8bdf8fc5d (table test-table [id=298c0d1972c24689a41dd7ca257363c0]) successfully deleted
I20260806 22:49:57.402056 31952 ts_tablet_manager.cc:1918] T 3fb4e815adb84a509eb38ce8bdf8fc5d P 9edf436137414f54b26af5a8ec47d81c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 22:49:57.402107 31952 log.cc:1196] T 3fb4e815adb84a509eb38ce8bdf8fc5d P 9edf436137414f54b26af5a8ec47d81c: Deleting WAL directory at /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056586788339-29547-0/minicluster-data/ts-0/wal/wals/3fb4e815adb84a509eb38ce8bdf8fc5d
I20260806 22:49:57.402340 31952 ts_tablet_manager.cc:1939] T 3fb4e815adb84a509eb38ce8bdf8fc5d P 9edf436137414f54b26af5a8ec47d81c: Deleting consensus metadata
I20260806 22:49:57.402649 31320 catalog_manager.cc:4841] TS 9edf436137414f54b26af5a8ec47d81c (127.28.218.193:35231): tablet 3fb4e815adb84a509eb38ce8bdf8fc5d (table test-table [id=298c0d1972c24689a41dd7ca257363c0]) successfully deleted
Aug 06 22:49:57 dist-test-slave-1xrh krb5kdc[31289](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056597, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 22:49:57 dist-test-slave-1xrh krb5kdc[31289](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056597, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.28.218.254@KRBTEST.COM
I20260806 22:49:57.422302 31961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:57.414461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32930 (local address 127.28.218.254:38323)
0806 22:49:57.414631 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:57.414632 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:57.414643 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:49:57.414697 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:57.414697 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:57.414707 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:57.414764 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:57.415370 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:57.415928 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:57.416642 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:57.416848 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:57.416959 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:57.419774 (+ 2815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:57.419775 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:57.419776 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:49:57.419782 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:57.421243 (+ 1461us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:57.421614 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:57.421616 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:57.421616 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:57.421676 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:57.421893 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:57.421893 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:57.421893 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:57.421998 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:57.422074 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:57.422150 (+ 76us) server_negotiation.cc:299] Negotiation successful
0806 22:49:57.422210 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":94,"threads_started":1}
I20260806 22:49:57.423854 31334 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:32930:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:49:57.424026 31334 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:49:57.425557 31478 tablet_service.cc:1467] Processing CreateTablet for tablet ae982983db134fc5ad6691dc57bcc864 (DEFAULT_TABLE table=test-table [id=4dc03601ea0444728f5efccb18773af6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:49:57.425614 31774 tablet_service.cc:1467] Processing CreateTablet for tablet ae982983db134fc5ad6691dc57bcc864 (DEFAULT_TABLE table=test-table [id=4dc03601ea0444728f5efccb18773af6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:49:57.425597 31626 tablet_service.cc:1467] Processing CreateTablet for tablet ae982983db134fc5ad6691dc57bcc864 (DEFAULT_TABLE table=test-table [id=4dc03601ea0444728f5efccb18773af6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:49:57.425685 31478 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae982983db134fc5ad6691dc57bcc864. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:49:57.425716 31774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae982983db134fc5ad6691dc57bcc864. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:49:57.425725 31626 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae982983db134fc5ad6691dc57bcc864. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:49:57.427179 31963 tablet_bootstrap.cc:492] T ae982983db134fc5ad6691dc57bcc864 P 4eaa65e90f624ffaa081592d38dfb33f: Bootstrap starting.
I20260806 22:49:57.427357 31962 tablet_bootstrap.cc:492] T ae982983db134fc5ad6691dc57bcc864 P 8da8827f19c842f6bfae8a9ee27ef69a: Bootstrap starting.
I20260806 22:49:57.427587 31963 tablet_bootstrap.cc:654] T ae982983db134fc5ad6691dc57bcc864 P 4eaa65e90f624ffaa081592d38dfb33f: Neither blocks nor log segments found. Creating new log.
I20260806 22:49:57.427731 31962 tablet_bootstrap.cc:654] T ae982983db134fc5ad6691dc57bcc864 P 8da8827f19c842f6bfae8a9ee27ef69a: Neither blocks nor log segments found. Creating new log.
I20260806 22:49:57.427893 31964 tablet_bootstrap.cc:492] T ae982983db134fc5ad6691dc57bcc864 P 9edf436137414f54b26af5a8ec47d81c: Bootstrap starting.
I20260806 22:49:57.428205 31963 tablet_bootstrap.cc:492] T ae982983db134fc5ad6691dc57bcc864 P 4eaa65e90f624ffaa081592d38dfb33f: No bootstrap required, opened a new log
I20260806 22:49:57.428270 31963 ts_tablet_manager.cc:1397] T ae982983db134fc5ad6691dc57bcc864 P 4eaa65e90f624ffaa081592d38dfb33f: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 22:49:57.428339 31962 tablet_bootstrap.cc:492] T ae982983db134fc5ad6691dc57bcc864 P 8da8827f19c842f6bfae8a9ee27ef69a: No bootstrap required, opened a new log
I20260806 22:49:57.428341 31964 tablet_bootstrap.cc:654] T ae982983db134fc5ad6691dc57bcc864 P 9edf436137414f54b26af5a8ec47d81c: Neither blocks nor log segments found. Creating new log.
I20260806 22:49:57.428393 31962 ts_tablet_manager.cc:1397] T ae982983db134fc5ad6691dc57bcc864 P 8da8827f19c842f6bfae8a9ee27ef69a: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 22:49:57.428428 31963 raft_consensus.cc:348] T ae982983db134fc5ad6691dc57bcc864 P 4eaa65e90f624ffaa081592d38dfb33f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9edf436137414f54b26af5a8ec47d81c" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 35231 } } peers { permanent_uuid: "8da8827f19c842f6bfae8a9ee27ef69a" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 33853 } } peers { permanent_uuid: "4eaa65e90f624ffaa081592d38dfb33f" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 41453 } }
I20260806 22:49:57.428534 31963 raft_consensus.cc:374] T ae982983db134fc5ad6691dc57bcc864 P 4eaa65e90f624ffaa081592d38dfb33f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:49:57.428509 31962 raft_consensus.cc:348] T ae982983db134fc5ad6691dc57bcc864 P 8da8827f19c842f6bfae8a9ee27ef69a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9edf436137414f54b26af5a8ec47d81c" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 35231 } } peers { permanent_uuid: "8da8827f19c842f6bfae8a9ee27ef69a" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 33853 } } peers { permanent_uuid: "4eaa65e90f624ffaa081592d38dfb33f" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 41453 } }
I20260806 22:49:57.428566 31963 raft_consensus.cc:729] T ae982983db134fc5ad6691dc57bcc864 P 4eaa65e90f624ffaa081592d38dfb33f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4eaa65e90f624ffaa081592d38dfb33f, State: Initialized, Role: FOLLOWER
I20260806 22:49:57.428572 31962 raft_consensus.cc:374] T ae982983db134fc5ad6691dc57bcc864 P 8da8827f19c842f6bfae8a9ee27ef69a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:49:57.428603 31962 raft_consensus.cc:729] T ae982983db134fc5ad6691dc57bcc864 P 8da8827f19c842f6bfae8a9ee27ef69a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8da8827f19c842f6bfae8a9ee27ef69a, State: Initialized, Role: FOLLOWER
I20260806 22:49:57.428650 31962 consensus_queue.cc:260] T ae982983db134fc5ad6691dc57bcc864 P 8da8827f19c842f6bfae8a9ee27ef69a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9edf436137414f54b26af5a8ec47d81c" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 35231 } } peers { permanent_uuid: "8da8827f19c842f6bfae8a9ee27ef69a" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 33853 } } peers { permanent_uuid: "4eaa65e90f624ffaa081592d38dfb33f" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 41453 } }
I20260806 22:49:57.428637 31963 consensus_queue.cc:260] T ae982983db134fc5ad6691dc57bcc864 P 4eaa65e90f624ffaa081592d38dfb33f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9edf436137414f54b26af5a8ec47d81c" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 35231 } } peers { permanent_uuid: "8da8827f19c842f6bfae8a9ee27ef69a" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 33853 } } peers { permanent_uuid: "4eaa65e90f624ffaa081592d38dfb33f" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 41453 } }
I20260806 22:49:57.428757 31962 ts_tablet_manager.cc:1428] T ae982983db134fc5ad6691dc57bcc864 P 8da8827f19c842f6bfae8a9ee27ef69a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:57.428962 31963 ts_tablet_manager.cc:1428] T ae982983db134fc5ad6691dc57bcc864 P 4eaa65e90f624ffaa081592d38dfb33f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 22:49:57.429484 31964 tablet_bootstrap.cc:492] T ae982983db134fc5ad6691dc57bcc864 P 9edf436137414f54b26af5a8ec47d81c: No bootstrap required, opened a new log
I20260806 22:49:57.429543 31964 ts_tablet_manager.cc:1397] T ae982983db134fc5ad6691dc57bcc864 P 9edf436137414f54b26af5a8ec47d81c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:49:57.429697 31964 raft_consensus.cc:348] T ae982983db134fc5ad6691dc57bcc864 P 9edf436137414f54b26af5a8ec47d81c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9edf436137414f54b26af5a8ec47d81c" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 35231 } } peers { permanent_uuid: "8da8827f19c842f6bfae8a9ee27ef69a" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 33853 } } peers { permanent_uuid: "4eaa65e90f624ffaa081592d38dfb33f" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 41453 } }
I20260806 22:49:57.429772 31964 raft_consensus.cc:374] T ae982983db134fc5ad6691dc57bcc864 P 9edf436137414f54b26af5a8ec47d81c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:49:57.429798 31964 raft_consensus.cc:729] T ae982983db134fc5ad6691dc57bcc864 P 9edf436137414f54b26af5a8ec47d81c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9edf436137414f54b26af5a8ec47d81c, State: Initialized, Role: FOLLOWER
I20260806 22:49:57.429845 31964 consensus_queue.cc:260] T ae982983db134fc5ad6691dc57bcc864 P 9edf436137414f54b26af5a8ec47d81c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9edf436137414f54b26af5a8ec47d81c" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 35231 } } peers { permanent_uuid: "8da8827f19c842f6bfae8a9ee27ef69a" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 33853 } } peers { permanent_uuid: "4eaa65e90f624ffaa081592d38dfb33f" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 41453 } }
I20260806 22:49:57.429977 31964 ts_tablet_manager.cc:1428] T ae982983db134fc5ad6691dc57bcc864 P 9edf436137414f54b26af5a8ec47d81c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:57.483902 31936 raft_consensus.cc:482] T ae982983db134fc5ad6691dc57bcc864 P 8da8827f19c842f6bfae8a9ee27ef69a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:49:57.484022 31936 raft_consensus.cc:504] T ae982983db134fc5ad6691dc57bcc864 P 8da8827f19c842f6bfae8a9ee27ef69a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9edf436137414f54b26af5a8ec47d81c" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 35231 } } peers { permanent_uuid: "8da8827f19c842f6bfae8a9ee27ef69a" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 33853 } } peers { permanent_uuid: "4eaa65e90f624ffaa081592d38dfb33f" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 41453 } }
I20260806 22:49:57.484198 31936 leader_election.cc:288] T ae982983db134fc5ad6691dc57bcc864 P 8da8827f19c842f6bfae8a9ee27ef69a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9edf436137414f54b26af5a8ec47d81c (127.28.218.193:35231), 4eaa65e90f624ffaa081592d38dfb33f (127.28.218.195:41453)
I20260806 22:49:57.484375 31501 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae982983db134fc5ad6691dc57bcc864" candidate_uuid: "8da8827f19c842f6bfae8a9ee27ef69a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9edf436137414f54b26af5a8ec47d81c" is_pre_election: true
I20260806 22:49:57.484447 31795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae982983db134fc5ad6691dc57bcc864" candidate_uuid: "8da8827f19c842f6bfae8a9ee27ef69a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4eaa65e90f624ffaa081592d38dfb33f" is_pre_election: true
I20260806 22:49:57.484494 31501 raft_consensus.cc:2454] T ae982983db134fc5ad6691dc57bcc864 P 9edf436137414f54b26af5a8ec47d81c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8da8827f19c842f6bfae8a9ee27ef69a in term 0.
I20260806 22:49:57.484543 31795 raft_consensus.cc:2454] T ae982983db134fc5ad6691dc57bcc864 P 4eaa65e90f624ffaa081592d38dfb33f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8da8827f19c842f6bfae8a9ee27ef69a in term 0.
I20260806 22:49:57.484654 31576 leader_election.cc:302] T ae982983db134fc5ad6691dc57bcc864 P 8da8827f19c842f6bfae8a9ee27ef69a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8da8827f19c842f6bfae8a9ee27ef69a, 9edf436137414f54b26af5a8ec47d81c; no voters:
I20260806 22:49:57.484746 31936 raft_consensus.cc:2781] T ae982983db134fc5ad6691dc57bcc864 P 8da8827f19c842f6bfae8a9ee27ef69a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:49:57.484817 31936 raft_consensus.cc:482] T ae982983db134fc5ad6691dc57bcc864 P 8da8827f19c842f6bfae8a9ee27ef69a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:49:57.484843 31936 raft_consensus.cc:3037] T ae982983db134fc5ad6691dc57bcc864 P 8da8827f19c842f6bfae8a9ee27ef69a [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:49:57.485553 31936 raft_consensus.cc:504] T ae982983db134fc5ad6691dc57bcc864 P 8da8827f19c842f6bfae8a9ee27ef69a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9edf436137414f54b26af5a8ec47d81c" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 35231 } } peers { permanent_uuid: "8da8827f19c842f6bfae8a9ee27ef69a" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 33853 } } peers { permanent_uuid: "4eaa65e90f624ffaa081592d38dfb33f" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 41453 } }
I20260806 22:49:57.485700 31936 leader_election.cc:288] T ae982983db134fc5ad6691dc57bcc864 P 8da8827f19c842f6bfae8a9ee27ef69a [CANDIDATE]: Term 1 election: Requested vote from peers 9edf436137414f54b26af5a8ec47d81c (127.28.218.193:35231), 4eaa65e90f624ffaa081592d38dfb33f (127.28.218.195:41453)
I20260806 22:49:57.485831 31795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae982983db134fc5ad6691dc57bcc864" candidate_uuid: "8da8827f19c842f6bfae8a9ee27ef69a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4eaa65e90f624ffaa081592d38dfb33f"
I20260806 22:49:57.485872 31501 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae982983db134fc5ad6691dc57bcc864" candidate_uuid: "8da8827f19c842f6bfae8a9ee27ef69a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9edf436137414f54b26af5a8ec47d81c"
I20260806 22:49:57.485924 31795 raft_consensus.cc:3037] T ae982983db134fc5ad6691dc57bcc864 P 4eaa65e90f624ffaa081592d38dfb33f [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:49:57.485960 31501 raft_consensus.cc:3037] T ae982983db134fc5ad6691dc57bcc864 P 9edf436137414f54b26af5a8ec47d81c [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:49:57.486545 31795 raft_consensus.cc:2454] T ae982983db134fc5ad6691dc57bcc864 P 4eaa65e90f624ffaa081592d38dfb33f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8da8827f19c842f6bfae8a9ee27ef69a in term 1.
I20260806 22:49:57.486606 31501 raft_consensus.cc:2454] T ae982983db134fc5ad6691dc57bcc864 P 9edf436137414f54b26af5a8ec47d81c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8da8827f19c842f6bfae8a9ee27ef69a in term 1.
I20260806 22:49:57.486759 31578 leader_election.cc:302] T ae982983db134fc5ad6691dc57bcc864 P 8da8827f19c842f6bfae8a9ee27ef69a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4eaa65e90f624ffaa081592d38dfb33f, 8da8827f19c842f6bfae8a9ee27ef69a; no voters:
I20260806 22:49:57.486871 31936 raft_consensus.cc:2781] T ae982983db134fc5ad6691dc57bcc864 P 8da8827f19c842f6bfae8a9ee27ef69a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:49:57.486917 31936 raft_consensus.cc:686] T ae982983db134fc5ad6691dc57bcc864 P 8da8827f19c842f6bfae8a9ee27ef69a [term 1 LEADER]: Becoming Leader. State: Replica: 8da8827f19c842f6bfae8a9ee27ef69a, State: Running, Role: LEADER
I20260806 22:49:57.486958 31936 consensus_queue.cc:237] T ae982983db134fc5ad6691dc57bcc864 P 8da8827f19c842f6bfae8a9ee27ef69a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9edf436137414f54b26af5a8ec47d81c" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 35231 } } peers { permanent_uuid: "8da8827f19c842f6bfae8a9ee27ef69a" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 33853 } } peers { permanent_uuid: "4eaa65e90f624ffaa081592d38dfb33f" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 41453 } }
I20260806 22:49:57.487646 31333 catalog_manager.cc:5495] T ae982983db134fc5ad6691dc57bcc864 P 8da8827f19c842f6bfae8a9ee27ef69a reported cstate change: term changed from 0 to 1, leader changed from <none> to 8da8827f19c842f6bfae8a9ee27ef69a (127.28.218.194). New cstate: current_term: 1 leader_uuid: "8da8827f19c842f6bfae8a9ee27ef69a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9edf436137414f54b26af5a8ec47d81c" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 35231 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8da8827f19c842f6bfae8a9ee27ef69a" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 33853 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4eaa65e90f624ffaa081592d38dfb33f" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 41453 } health_report { overall_health: UNKNOWN } } }
I20260806 22:49:57.504906 31972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:57.502738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60726 (local address 127.28.218.194:33853)
0806 22:49:57.502890 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:57.502892 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:57.502899 (+ 7us) server_negotiation.cc:407] Connection header received
0806 22:49:57.502917 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:57.502917 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:57.502926 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:57.502998 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:49:57.503116 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:57.503637 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:57.504359 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:57.504547 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:57.504599 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:57.504702 (+ 103us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:49:57.504726 (+ 24us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:49:57.504759 (+ 33us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:57.504790 (+ 31us) server_negotiation.cc:299] Negotiation successful
0806 22:49:57.504805 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":97,"threads_started":1}
I20260806 22:49:57.506853 31501 raft_consensus.cc:1261] T ae982983db134fc5ad6691dc57bcc864 P 9edf436137414f54b26af5a8ec47d81c [term 1 FOLLOWER]: Refusing update from remote peer 8da8827f19c842f6bfae8a9ee27ef69a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:49:57.506845 31795 raft_consensus.cc:1261] T ae982983db134fc5ad6691dc57bcc864 P 4eaa65e90f624ffaa081592d38dfb33f [term 1 FOLLOWER]: Refusing update from remote peer 8da8827f19c842f6bfae8a9ee27ef69a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:49:57.507074 31898 consensus_queue.cc:1035] T ae982983db134fc5ad6691dc57bcc864 P 8da8827f19c842f6bfae8a9ee27ef69a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9edf436137414f54b26af5a8ec47d81c" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 35231 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:49:57.507184 31930 consensus_queue.cc:1035] T ae982983db134fc5ad6691dc57bcc864 P 8da8827f19c842f6bfae8a9ee27ef69a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4eaa65e90f624ffaa081592d38dfb33f" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 41453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:49:57.516986 31333 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:32930:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:49:57.517063 31333 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:32930:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:49:57.517608 31333 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d5a2faa728f142b79312f4cb47b13e55: Sending DeleteTablet for 3 replicas of tablet ae982983db134fc5ad6691dc57bcc864
I20260806 22:49:57.517849 31478 tablet_service.cc:1514] Processing DeleteTablet for tablet ae982983db134fc5ad6691dc57bcc864 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:49:57 UTC) from {username='kudu', principal='kudu/127.28.218.254@KRBTEST.COM'} at 127.0.0.1:41908
I20260806 22:49:57.517868 31774 tablet_service.cc:1514] Processing DeleteTablet for tablet ae982983db134fc5ad6691dc57bcc864 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:49:57 UTC) from {username='kudu', principal='kudu/127.28.218.254@KRBTEST.COM'} at 127.0.0.1:60258
I20260806 22:49:57.517954 31953 tablet_replica.cc:330] T ae982983db134fc5ad6691dc57bcc864 P 4eaa65e90f624ffaa081592d38dfb33f: stopping tablet replica
I20260806 22:49:57.517954 31952 tablet_replica.cc:330] T ae982983db134fc5ad6691dc57bcc864 P 9edf436137414f54b26af5a8ec47d81c: stopping tablet replica
I20260806 22:49:57.517957 31626 tablet_service.cc:1514] Processing DeleteTablet for tablet ae982983db134fc5ad6691dc57bcc864 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:49:57 UTC) from {username='kudu', principal='kudu/127.28.218.254@KRBTEST.COM'} at 127.0.0.1:60708
I20260806 22:49:57.518003 31953 raft_consensus.cc:2229] T ae982983db134fc5ad6691dc57bcc864 P 4eaa65e90f624ffaa081592d38dfb33f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:49:57.518009 31952 raft_consensus.cc:2229] T ae982983db134fc5ad6691dc57bcc864 P 9edf436137414f54b26af5a8ec47d81c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:49:57.518028 31951 tablet_replica.cc:330] T ae982983db134fc5ad6691dc57bcc864 P 8da8827f19c842f6bfae8a9ee27ef69a: stopping tablet replica
I20260806 22:49:57.518042 31953 raft_consensus.cc:2258] T ae982983db134fc5ad6691dc57bcc864 P 4eaa65e90f624ffaa081592d38dfb33f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:49:57.518045 31952 raft_consensus.cc:2258] T ae982983db134fc5ad6691dc57bcc864 P 9edf436137414f54b26af5a8ec47d81c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:49:57.518076 31951 raft_consensus.cc:2229] T ae982983db134fc5ad6691dc57bcc864 P 8da8827f19c842f6bfae8a9ee27ef69a [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:49:57.518150 31951 raft_consensus.cc:2258] T ae982983db134fc5ad6691dc57bcc864 P 8da8827f19c842f6bfae8a9ee27ef69a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:49:57.518325 31953 ts_tablet_manager.cc:1905] T ae982983db134fc5ad6691dc57bcc864 P 4eaa65e90f624ffaa081592d38dfb33f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:49:57.518433 31951 ts_tablet_manager.cc:1905] T ae982983db134fc5ad6691dc57bcc864 P 8da8827f19c842f6bfae8a9ee27ef69a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:49:57.518392 31333 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:32930:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:49:57.518515 31333 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:49:57.519730 31953 ts_tablet_manager.cc:1918] T ae982983db134fc5ad6691dc57bcc864 P 4eaa65e90f624ffaa081592d38dfb33f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 22:49:57.519793 31953 log.cc:1196] T ae982983db134fc5ad6691dc57bcc864 P 4eaa65e90f624ffaa081592d38dfb33f: Deleting WAL directory at /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056586788339-29547-0/minicluster-data/ts-2/wal/wals/ae982983db134fc5ad6691dc57bcc864
I20260806 22:49:57.519868 31774 tablet_service.cc:1467] Processing CreateTablet for tablet 4aa4dd5127724d64816ebd3840c9d658 (DEFAULT_TABLE table=test-table [id=7cedc88809e54d66b553138d5d78a770]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:49:57.519903 31952 ts_tablet_manager.cc:1905] T ae982983db134fc5ad6691dc57bcc864 P 9edf436137414f54b26af5a8ec47d81c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:49:57.519972 31774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4aa4dd5127724d64816ebd3840c9d658. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:49:57.520048 31953 ts_tablet_manager.cc:1939] T ae982983db134fc5ad6691dc57bcc864 P 4eaa65e90f624ffaa081592d38dfb33f: Deleting consensus metadata
I20260806 22:49:57.520114 31951 ts_tablet_manager.cc:1918] T ae982983db134fc5ad6691dc57bcc864 P 8da8827f19c842f6bfae8a9ee27ef69a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 22:49:57.520159 31951 log.cc:1196] T ae982983db134fc5ad6691dc57bcc864 P 8da8827f19c842f6bfae8a9ee27ef69a: Deleting WAL directory at /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056586788339-29547-0/minicluster-data/ts-1/wal/wals/ae982983db134fc5ad6691dc57bcc864
I20260806 22:49:57.520378 31951 ts_tablet_manager.cc:1939] T ae982983db134fc5ad6691dc57bcc864 P 8da8827f19c842f6bfae8a9ee27ef69a: Deleting consensus metadata
I20260806 22:49:57.520637 31322 catalog_manager.cc:4841] TS 4eaa65e90f624ffaa081592d38dfb33f (127.28.218.195:41453): tablet ae982983db134fc5ad6691dc57bcc864 (table test-table [id=4dc03601ea0444728f5efccb18773af6]) successfully deleted
I20260806 22:49:57.521162 31963 tablet_bootstrap.cc:492] T 4aa4dd5127724d64816ebd3840c9d658 P 4eaa65e90f624ffaa081592d38dfb33f: Bootstrap starting.
I20260806 22:49:57.521418 31321 catalog_manager.cc:4841] TS 8da8827f19c842f6bfae8a9ee27ef69a (127.28.218.194:33853): tablet ae982983db134fc5ad6691dc57bcc864 (table test-table [id=4dc03601ea0444728f5efccb18773af6]) successfully deleted
I20260806 22:49:57.521461 31626 tablet_service.cc:1467] Processing CreateTablet for tablet 4aa4dd5127724d64816ebd3840c9d658 (DEFAULT_TABLE table=test-table [id=7cedc88809e54d66b553138d5d78a770]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:49:57.521514 31952 ts_tablet_manager.cc:1918] T ae982983db134fc5ad6691dc57bcc864 P 9edf436137414f54b26af5a8ec47d81c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 22:49:57.521553 31626 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4aa4dd5127724d64816ebd3840c9d658. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:49:57.521560 31952 log.cc:1196] T ae982983db134fc5ad6691dc57bcc864 P 9edf436137414f54b26af5a8ec47d81c: Deleting WAL directory at /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056586788339-29547-0/minicluster-data/ts-0/wal/wals/ae982983db134fc5ad6691dc57bcc864
I20260806 22:49:57.521687 31478 tablet_service.cc:1467] Processing CreateTablet for tablet 4aa4dd5127724d64816ebd3840c9d658 (DEFAULT_TABLE table=test-table [id=7cedc88809e54d66b553138d5d78a770]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:49:57.521729 31963 tablet_bootstrap.cc:654] T 4aa4dd5127724d64816ebd3840c9d658 P 4eaa65e90f624ffaa081592d38dfb33f: Neither blocks nor log segments found. Creating new log.
I20260806 22:49:57.521770 31952 ts_tablet_manager.cc:1939] T ae982983db134fc5ad6691dc57bcc864 P 9edf436137414f54b26af5a8ec47d81c: Deleting consensus metadata
I20260806 22:49:57.521786 31478 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4aa4dd5127724d64816ebd3840c9d658. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:49:57.522619 31962 tablet_bootstrap.cc:492] T 4aa4dd5127724d64816ebd3840c9d658 P 8da8827f19c842f6bfae8a9ee27ef69a: Bootstrap starting.
I20260806 22:49:57.522895 31962 tablet_bootstrap.cc:654] T 4aa4dd5127724d64816ebd3840c9d658 P 8da8827f19c842f6bfae8a9ee27ef69a: Neither blocks nor log segments found. Creating new log.
I20260806 22:49:57.522933 31320 catalog_manager.cc:4841] TS 9edf436137414f54b26af5a8ec47d81c (127.28.218.193:35231): tablet ae982983db134fc5ad6691dc57bcc864 (table test-table [id=4dc03601ea0444728f5efccb18773af6]) successfully deleted
I20260806 22:49:57.523150 31964 tablet_bootstrap.cc:492] T 4aa4dd5127724d64816ebd3840c9d658 P 9edf436137414f54b26af5a8ec47d81c: Bootstrap starting.
I20260806 22:49:57.523418 31962 tablet_bootstrap.cc:492] T 4aa4dd5127724d64816ebd3840c9d658 P 8da8827f19c842f6bfae8a9ee27ef69a: No bootstrap required, opened a new log
I20260806 22:49:57.523458 31962 ts_tablet_manager.cc:1397] T 4aa4dd5127724d64816ebd3840c9d658 P 8da8827f19c842f6bfae8a9ee27ef69a: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 22:49:57.523442 31964 tablet_bootstrap.cc:654] T 4aa4dd5127724d64816ebd3840c9d658 P 9edf436137414f54b26af5a8ec47d81c: Neither blocks nor log segments found. Creating new log.
I20260806 22:49:57.523559 31962 raft_consensus.cc:348] T 4aa4dd5127724d64816ebd3840c9d658 P 8da8827f19c842f6bfae8a9ee27ef69a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9edf436137414f54b26af5a8ec47d81c" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 35231 } } peers { permanent_uuid: "8da8827f19c842f6bfae8a9ee27ef69a" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 33853 } } peers { permanent_uuid: "4eaa65e90f624ffaa081592d38dfb33f" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 41453 } }
I20260806 22:49:57.523618 31962 raft_consensus.cc:374] T 4aa4dd5127724d64816ebd3840c9d658 P 8da8827f19c842f6bfae8a9ee27ef69a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:49:57.523633 31962 raft_consensus.cc:729] T 4aa4dd5127724d64816ebd3840c9d658 P 8da8827f19c842f6bfae8a9ee27ef69a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8da8827f19c842f6bfae8a9ee27ef69a, State: Initialized, Role: FOLLOWER
I20260806 22:49:57.523679 31962 consensus_queue.cc:260] T 4aa4dd5127724d64816ebd3840c9d658 P 8da8827f19c842f6bfae8a9ee27ef69a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9edf436137414f54b26af5a8ec47d81c" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 35231 } } peers { permanent_uuid: "8da8827f19c842f6bfae8a9ee27ef69a" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 33853 } } peers { permanent_uuid: "4eaa65e90f624ffaa081592d38dfb33f" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 41453 } }
I20260806 22:49:57.523772 31962 ts_tablet_manager.cc:1428] T 4aa4dd5127724d64816ebd3840c9d658 P 8da8827f19c842f6bfae8a9ee27ef69a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:57.523834 31963 tablet_bootstrap.cc:492] T 4aa4dd5127724d64816ebd3840c9d658 P 4eaa65e90f624ffaa081592d38dfb33f: No bootstrap required, opened a new log
I20260806 22:49:57.523902 31963 ts_tablet_manager.cc:1397] T 4aa4dd5127724d64816ebd3840c9d658 P 4eaa65e90f624ffaa081592d38dfb33f: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260806 22:49:57.524053 31963 raft_consensus.cc:348] T 4aa4dd5127724d64816ebd3840c9d658 P 4eaa65e90f624ffaa081592d38dfb33f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9edf436137414f54b26af5a8ec47d81c" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 35231 } } peers { permanent_uuid: "8da8827f19c842f6bfae8a9ee27ef69a" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 33853 } } peers { permanent_uuid: "4eaa65e90f624ffaa081592d38dfb33f" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 41453 } }
I20260806 22:49:57.524129 31963 raft_consensus.cc:374] T 4aa4dd5127724d64816ebd3840c9d658 P 4eaa65e90f624ffaa081592d38dfb33f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:49:57.524153 31963 raft_consensus.cc:729] T 4aa4dd5127724d64816ebd3840c9d658 P 4eaa65e90f624ffaa081592d38dfb33f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4eaa65e90f624ffaa081592d38dfb33f, State: Initialized, Role: FOLLOWER
I20260806 22:49:57.524196 31963 consensus_queue.cc:260] T 4aa4dd5127724d64816ebd3840c9d658 P 4eaa65e90f624ffaa081592d38dfb33f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9edf436137414f54b26af5a8ec47d81c" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 35231 } } peers { permanent_uuid: "8da8827f19c842f6bfae8a9ee27ef69a" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 33853 } } peers { permanent_uuid: "4eaa65e90f624ffaa081592d38dfb33f" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 41453 } }
I20260806 22:49:57.524282 31964 tablet_bootstrap.cc:492] T 4aa4dd5127724d64816ebd3840c9d658 P 9edf436137414f54b26af5a8ec47d81c: No bootstrap required, opened a new log
I20260806 22:49:57.524295 31963 ts_tablet_manager.cc:1428] T 4aa4dd5127724d64816ebd3840c9d658 P 4eaa65e90f624ffaa081592d38dfb33f: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:57.524334 31964 ts_tablet_manager.cc:1397] T 4aa4dd5127724d64816ebd3840c9d658 P 9edf436137414f54b26af5a8ec47d81c: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 22:49:57.524446 31964 raft_consensus.cc:348] T 4aa4dd5127724d64816ebd3840c9d658 P 9edf436137414f54b26af5a8ec47d81c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9edf436137414f54b26af5a8ec47d81c" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 35231 } } peers { permanent_uuid: "8da8827f19c842f6bfae8a9ee27ef69a" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 33853 } } peers { permanent_uuid: "4eaa65e90f624ffaa081592d38dfb33f" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 41453 } }
I20260806 22:49:57.524542 31964 raft_consensus.cc:374] T 4aa4dd5127724d64816ebd3840c9d658 P 9edf436137414f54b26af5a8ec47d81c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:49:57.524569 31964 raft_consensus.cc:729] T 4aa4dd5127724d64816ebd3840c9d658 P 9edf436137414f54b26af5a8ec47d81c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9edf436137414f54b26af5a8ec47d81c, State: Initialized, Role: FOLLOWER
I20260806 22:49:57.524627 31964 consensus_queue.cc:260] T 4aa4dd5127724d64816ebd3840c9d658 P 9edf436137414f54b26af5a8ec47d81c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9edf436137414f54b26af5a8ec47d81c" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 35231 } } peers { permanent_uuid: "8da8827f19c842f6bfae8a9ee27ef69a" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 33853 } } peers { permanent_uuid: "4eaa65e90f624ffaa081592d38dfb33f" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 41453 } }
I20260806 22:49:57.524725 31964 ts_tablet_manager.cc:1428] T 4aa4dd5127724d64816ebd3840c9d658 P 9edf436137414f54b26af5a8ec47d81c: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260806 22:49:57.657181 31936 raft_consensus.cc:482] T 4aa4dd5127724d64816ebd3840c9d658 P 8da8827f19c842f6bfae8a9ee27ef69a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:49:57.657274 31936 raft_consensus.cc:504] T 4aa4dd5127724d64816ebd3840c9d658 P 8da8827f19c842f6bfae8a9ee27ef69a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9edf436137414f54b26af5a8ec47d81c" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 35231 } } peers { permanent_uuid: "8da8827f19c842f6bfae8a9ee27ef69a" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 33853 } } peers { permanent_uuid: "4eaa65e90f624ffaa081592d38dfb33f" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 41453 } }
I20260806 22:49:57.657462 31936 leader_election.cc:288] T 4aa4dd5127724d64816ebd3840c9d658 P 8da8827f19c842f6bfae8a9ee27ef69a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9edf436137414f54b26af5a8ec47d81c (127.28.218.193:35231), 4eaa65e90f624ffaa081592d38dfb33f (127.28.218.195:41453)
I20260806 22:49:57.657675 31501 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4aa4dd5127724d64816ebd3840c9d658" candidate_uuid: "8da8827f19c842f6bfae8a9ee27ef69a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9edf436137414f54b26af5a8ec47d81c" is_pre_election: true
I20260806 22:49:57.657701 31795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4aa4dd5127724d64816ebd3840c9d658" candidate_uuid: "8da8827f19c842f6bfae8a9ee27ef69a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4eaa65e90f624ffaa081592d38dfb33f" is_pre_election: true
I20260806 22:49:57.657794 31501 raft_consensus.cc:2454] T 4aa4dd5127724d64816ebd3840c9d658 P 9edf436137414f54b26af5a8ec47d81c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8da8827f19c842f6bfae8a9ee27ef69a in term 0.
I20260806 22:49:57.657790 31795 raft_consensus.cc:2454] T 4aa4dd5127724d64816ebd3840c9d658 P 4eaa65e90f624ffaa081592d38dfb33f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8da8827f19c842f6bfae8a9ee27ef69a in term 0.
I20260806 22:49:57.657989 31578 leader_election.cc:302] T 4aa4dd5127724d64816ebd3840c9d658 P 8da8827f19c842f6bfae8a9ee27ef69a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4eaa65e90f624ffaa081592d38dfb33f, 8da8827f19c842f6bfae8a9ee27ef69a; no voters:
I20260806 22:49:57.658087 31936 raft_consensus.cc:2781] T 4aa4dd5127724d64816ebd3840c9d658 P 8da8827f19c842f6bfae8a9ee27ef69a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:49:57.658131 31936 raft_consensus.cc:482] T 4aa4dd5127724d64816ebd3840c9d658 P 8da8827f19c842f6bfae8a9ee27ef69a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:49:57.658151 31936 raft_consensus.cc:3037] T 4aa4dd5127724d64816ebd3840c9d658 P 8da8827f19c842f6bfae8a9ee27ef69a [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:49:57.658780 31936 raft_consensus.cc:504] T 4aa4dd5127724d64816ebd3840c9d658 P 8da8827f19c842f6bfae8a9ee27ef69a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9edf436137414f54b26af5a8ec47d81c" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 35231 } } peers { permanent_uuid: "8da8827f19c842f6bfae8a9ee27ef69a" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 33853 } } peers { permanent_uuid: "4eaa65e90f624ffaa081592d38dfb33f" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 41453 } }
I20260806 22:49:57.658962 31936 leader_election.cc:288] T 4aa4dd5127724d64816ebd3840c9d658 P 8da8827f19c842f6bfae8a9ee27ef69a [CANDIDATE]: Term 1 election: Requested vote from peers 9edf436137414f54b26af5a8ec47d81c (127.28.218.193:35231), 4eaa65e90f624ffaa081592d38dfb33f (127.28.218.195:41453)
I20260806 22:49:57.659159 31795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4aa4dd5127724d64816ebd3840c9d658" candidate_uuid: "8da8827f19c842f6bfae8a9ee27ef69a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4eaa65e90f624ffaa081592d38dfb33f"
I20260806 22:49:57.659158 31501 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4aa4dd5127724d64816ebd3840c9d658" candidate_uuid: "8da8827f19c842f6bfae8a9ee27ef69a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9edf436137414f54b26af5a8ec47d81c"
I20260806 22:49:57.659235 31795 raft_consensus.cc:3037] T 4aa4dd5127724d64816ebd3840c9d658 P 4eaa65e90f624ffaa081592d38dfb33f [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:49:57.659252 31501 raft_consensus.cc:3037] T 4aa4dd5127724d64816ebd3840c9d658 P 9edf436137414f54b26af5a8ec47d81c [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:49:57.659860 31501 raft_consensus.cc:2454] T 4aa4dd5127724d64816ebd3840c9d658 P 9edf436137414f54b26af5a8ec47d81c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8da8827f19c842f6bfae8a9ee27ef69a in term 1.
I20260806 22:49:57.659910 31795 raft_consensus.cc:2454] T 4aa4dd5127724d64816ebd3840c9d658 P 4eaa65e90f624ffaa081592d38dfb33f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8da8827f19c842f6bfae8a9ee27ef69a in term 1.
I20260806 22:49:57.660076 31576 leader_election.cc:302] T 4aa4dd5127724d64816ebd3840c9d658 P 8da8827f19c842f6bfae8a9ee27ef69a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8da8827f19c842f6bfae8a9ee27ef69a, 9edf436137414f54b26af5a8ec47d81c; no voters:
I20260806 22:49:57.660211 31936 raft_consensus.cc:2781] T 4aa4dd5127724d64816ebd3840c9d658 P 8da8827f19c842f6bfae8a9ee27ef69a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:49:57.660290 31936 raft_consensus.cc:686] T 4aa4dd5127724d64816ebd3840c9d658 P 8da8827f19c842f6bfae8a9ee27ef69a [term 1 LEADER]: Becoming Leader. State: Replica: 8da8827f19c842f6bfae8a9ee27ef69a, State: Running, Role: LEADER
I20260806 22:49:57.660346 31936 consensus_queue.cc:237] T 4aa4dd5127724d64816ebd3840c9d658 P 8da8827f19c842f6bfae8a9ee27ef69a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9edf436137414f54b26af5a8ec47d81c" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 35231 } } peers { permanent_uuid: "8da8827f19c842f6bfae8a9ee27ef69a" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 33853 } } peers { permanent_uuid: "4eaa65e90f624ffaa081592d38dfb33f" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 41453 } }
I20260806 22:49:57.660944 31334 catalog_manager.cc:5495] T 4aa4dd5127724d64816ebd3840c9d658 P 8da8827f19c842f6bfae8a9ee27ef69a reported cstate change: term changed from 0 to 1, leader changed from <none> to 8da8827f19c842f6bfae8a9ee27ef69a (127.28.218.194). New cstate: current_term: 1 leader_uuid: "8da8827f19c842f6bfae8a9ee27ef69a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9edf436137414f54b26af5a8ec47d81c" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 35231 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8da8827f19c842f6bfae8a9ee27ef69a" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 33853 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4eaa65e90f624ffaa081592d38dfb33f" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 41453 } health_report { overall_health: UNKNOWN } } }
I20260806 22:49:57.667972 31501 raft_consensus.cc:1261] T 4aa4dd5127724d64816ebd3840c9d658 P 9edf436137414f54b26af5a8ec47d81c [term 1 FOLLOWER]: Refusing update from remote peer 8da8827f19c842f6bfae8a9ee27ef69a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:49:57.668044 31795 raft_consensus.cc:1261] T 4aa4dd5127724d64816ebd3840c9d658 P 4eaa65e90f624ffaa081592d38dfb33f [term 1 FOLLOWER]: Refusing update from remote peer 8da8827f19c842f6bfae8a9ee27ef69a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:49:57.668154 31930 consensus_queue.cc:1035] T 4aa4dd5127724d64816ebd3840c9d658 P 8da8827f19c842f6bfae8a9ee27ef69a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9edf436137414f54b26af5a8ec47d81c" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 35231 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:49:57.668242 31936 consensus_queue.cc:1035] T 4aa4dd5127724d64816ebd3840c9d658 P 8da8827f19c842f6bfae8a9ee27ef69a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4eaa65e90f624ffaa081592d38dfb33f" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 41453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 22:49:57.672452 31928 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260806 22:49:59.191659 31334 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:32930:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:49:59.191753 31334 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:32930:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:49:59.192601 31334 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d5a2faa728f142b79312f4cb47b13e55: Sending DeleteTablet for 3 replicas of tablet 4aa4dd5127724d64816ebd3840c9d658
I20260806 22:49:59.192818 31478 tablet_service.cc:1514] Processing DeleteTablet for tablet 4aa4dd5127724d64816ebd3840c9d658 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:49:59 UTC) from {username='kudu', principal='kudu/127.28.218.254@KRBTEST.COM'} at 127.0.0.1:41908
I20260806 22:49:59.192829 31627 tablet_service.cc:1514] Processing DeleteTablet for tablet 4aa4dd5127724d64816ebd3840c9d658 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:49:59 UTC) from {username='kudu', principal='kudu/127.28.218.254@KRBTEST.COM'} at 127.0.0.1:60708
I20260806 22:49:59.192852 31774 tablet_service.cc:1514] Processing DeleteTablet for tablet 4aa4dd5127724d64816ebd3840c9d658 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:49:59 UTC) from {username='kudu', principal='kudu/127.28.218.254@KRBTEST.COM'} at 127.0.0.1:60258
I20260806 22:49:59.193043 32006 tablet_replica.cc:330] T 4aa4dd5127724d64816ebd3840c9d658 P 8da8827f19c842f6bfae8a9ee27ef69a: stopping tablet replica
I20260806 22:49:59.193053 32003 tablet_replica.cc:330] T 4aa4dd5127724d64816ebd3840c9d658 P 9edf436137414f54b26af5a8ec47d81c: stopping tablet replica
I20260806 22:49:59.193045 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 31410
I20260806 22:49:59.194128 32006 raft_consensus.cc:2229] T 4aa4dd5127724d64816ebd3840c9d658 P 8da8827f19c842f6bfae8a9ee27ef69a [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:49:59.194203 32006 raft_consensus.cc:2258] T 4aa4dd5127724d64816ebd3840c9d658 P 8da8827f19c842f6bfae8a9ee27ef69a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:49:59.194307 32004 tablet_replica.cc:330] T 4aa4dd5127724d64816ebd3840c9d658 P 4eaa65e90f624ffaa081592d38dfb33f: stopping tablet replica
I20260806 22:49:59.194384 32004 raft_consensus.cc:2229] T 4aa4dd5127724d64816ebd3840c9d658 P 4eaa65e90f624ffaa081592d38dfb33f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:49:59.194450 32004 raft_consensus.cc:2258] T 4aa4dd5127724d64816ebd3840c9d658 P 4eaa65e90f624ffaa081592d38dfb33f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:49:59.194489 32006 ts_tablet_manager.cc:1905] T 4aa4dd5127724d64816ebd3840c9d658 P 8da8827f19c842f6bfae8a9ee27ef69a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:49:59.194694 32004 ts_tablet_manager.cc:1905] T 4aa4dd5127724d64816ebd3840c9d658 P 4eaa65e90f624ffaa081592d38dfb33f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:49:59.196328 32006 ts_tablet_manager.cc:1918] T 4aa4dd5127724d64816ebd3840c9d658 P 8da8827f19c842f6bfae8a9ee27ef69a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 22:49:59.196447 32004 ts_tablet_manager.cc:1918] T 4aa4dd5127724d64816ebd3840c9d658 P 4eaa65e90f624ffaa081592d38dfb33f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 22:49:59.196478 32006 log.cc:1196] T 4aa4dd5127724d64816ebd3840c9d658 P 8da8827f19c842f6bfae8a9ee27ef69a: Deleting WAL directory at /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056586788339-29547-0/minicluster-data/ts-1/wal/wals/4aa4dd5127724d64816ebd3840c9d658
I20260806 22:49:59.196516 32004 log.cc:1196] T 4aa4dd5127724d64816ebd3840c9d658 P 4eaa65e90f624ffaa081592d38dfb33f: Deleting WAL directory at /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056586788339-29547-0/minicluster-data/ts-2/wal/wals/4aa4dd5127724d64816ebd3840c9d658
I20260806 22:49:59.196741 32004 ts_tablet_manager.cc:1939] T 4aa4dd5127724d64816ebd3840c9d658 P 4eaa65e90f624ffaa081592d38dfb33f: Deleting consensus metadata
I20260806 22:49:59.196741 32006 ts_tablet_manager.cc:1939] T 4aa4dd5127724d64816ebd3840c9d658 P 8da8827f19c842f6bfae8a9ee27ef69a: Deleting consensus metadata
I20260806 22:49:59.197034 31321 catalog_manager.cc:4841] TS 8da8827f19c842f6bfae8a9ee27ef69a (127.28.218.194:33853): tablet 4aa4dd5127724d64816ebd3840c9d658 (table test-table [id=7cedc88809e54d66b553138d5d78a770]) successfully deleted
I20260806 22:49:59.197067 31322 catalog_manager.cc:4841] TS 4eaa65e90f624ffaa081592d38dfb33f (127.28.218.195:41453): tablet 4aa4dd5127724d64816ebd3840c9d658 (table test-table [id=7cedc88809e54d66b553138d5d78a770]) successfully deleted
W20260806 22:49:59.201454 31320 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.28.218.193:35231 (error 108)
I20260806 22:49:59.201735 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 31559
I20260806 22:49:59.202898 32008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:59.201839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.193:35231 (local address 127.0.0.1:41924)
0806 22:49:59.202822 (+ 983us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:59.202865 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.193:35231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":938,"thread_start_us":56,"threads_started":1}
W20260806 22:49:59.203052 31320 catalog_manager.cc:4584] TS 9edf436137414f54b26af5a8ec47d81c (127.28.218.193:35231): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4aa4dd5127724d64816ebd3840c9d658: Network error: Client connection negotiation failed: client connection to 127.28.218.193:35231: connect: Connection refused (error 111)
I20260806 22:49:59.209543 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 31706
I20260806 22:49:59.215480 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 31302
2026-08-06T22:49:59Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (5912 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056586788339-29547-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:49:59 dist-test-slave-1xrh krb5kdc[32011](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:49:59 dist-test-slave-1xrh krb5kdc[32011](info): set up 2 sockets
Aug 06 22:49:59 dist-test-slave-1xrh krb5kdc[32011](info): commencing operation
krb5kdc: starting...
W20260806 22:50:00.279641 29547 mini_kdc.cc:121] Time spent starting KDC: real 1.042s user 0.004s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056586788339-29547-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056586788339-29547-0/krb5kdc/test-user.keytab.
Aug 06 22:50:00 dist-test-slave-1xrh krb5kdc[32011](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056600, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:50:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:50:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.218.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.218.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.218.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab.
Entry for principal kudu/127.28.218.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab.
I20260806 22:50:00.398504 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056586788339-29547-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056586788339-29547-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056586788339-29547-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.218.254:43941
--webserver_interface=127.28.218.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab
--principal=kudu/127.28.218.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.218.212:39411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.218.254:43941
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:50:00.478828 32026 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:50:00.479090 32026 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:50:00.479126 32026 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:50:00.480257 32026 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:50:00.480325 32026 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:50:00.480347 32026 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:50:00.480365 32026 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:50:00.480392 32026 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:50:00.481822 32026 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:39411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056586788339-29547-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056586788339-29547-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.218.254:43941
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab
--principal=kudu/127.28.218.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.218.254:43941
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.218.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32026
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056586788339-29547-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:50:00.482086 32026 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:50:00.482302 32026 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:50:00.487125 32034 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:00.487102 32031 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:00.487092 32032 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:50:00.487356 32026 server_base.cc:1034] running on GCE node
I20260806 22:50:00.487648 32026 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:50:00.487936 32026 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:50:00.489110 32026 hybrid_clock.cc:648] HybridClock initialized: now 1786056600489090 us; error 38 us; skew 500 ppm
Aug 06 22:50:00 dist-test-slave-1xrh krb5kdc[32011](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056600, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:50:00.496068 32026 init.cc:377] Logged in from keytab as kudu/127.28.218.254@KRBTEST.COM (short username kudu)
I20260806 22:50:00.496493 32026 webserver.cc:460] Webserver started at http://127.28.218.254:41577/ using document root <none> and password file <none>
I20260806 22:50:00.496739 32026 fs_manager.cc:359] Metadata directory not provided
I20260806 22:50:00.496810 32026 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:50:00.496958 32026 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:50:00.498008 32026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056586788339-29547-0/minicluster-data/master-0/data/instance:
uuid: "d753d7ad0f8946dda24d796c990f89cf"
format_stamp: "Formatted at 2026-08-06 22:50:00 on dist-test-slave-1xrh"
server_key: "c1c5baabd3e0da8762a7bd6c2ac1c6a7"
server_key_iv: "984f89b38bff80d5a04f98afa7fe0ee6"
server_key_version: "encryptionkey@0"
I20260806 22:50:00.498369 32026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056586788339-29547-0/minicluster-data/master-0/wal/instance:
uuid: "d753d7ad0f8946dda24d796c990f89cf"
format_stamp: "Formatted at 2026-08-06 22:50:00 on dist-test-slave-1xrh"
server_key: "c1c5baabd3e0da8762a7bd6c2ac1c6a7"
server_key_iv: "984f89b38bff80d5a04f98afa7fe0ee6"
server_key_version: "encryptionkey@0"
I20260806 22:50:00.499780 32026 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:50:00.500521 32041 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:00.500689 32026 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:50:00.500792 32026 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056586788339-29547-0/minicluster-data/master-0/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056586788339-29547-0/minicluster-data/master-0/wal
uuid: "d753d7ad0f8946dda24d796c990f89cf"
format_stamp: "Formatted at 2026-08-06 22:50:00 on dist-test-slave-1xrh"
server_key: "c1c5baabd3e0da8762a7bd6c2ac1c6a7"
server_key_iv: "984f89b38bff80d5a04f98afa7fe0ee6"
server_key_version: "encryptionkey@0"
I20260806 22:50:00.500864 32026 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056586788339-29547-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056586788339-29547-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056586788339-29547-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:50:00.511264 32026 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:50:00.513041 32026 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:50:00.513195 32026 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:50:00.517423 32026 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.254:43941
I20260806 22:50:00.517472 32093 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.254:43941 every 8 connection(s)
I20260806 22:50:00.517865 32026 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
I20260806 22:50:00.518698 32094 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:50:00.520949 32094 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d753d7ad0f8946dda24d796c990f89cf: Bootstrap starting.
I20260806 22:50:00.521517 32094 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d753d7ad0f8946dda24d796c990f89cf: Neither blocks nor log segments found. Creating new log.
I20260806 22:50:00.521798 32094 log.cc:824] T 00000000000000000000000000000000 P d753d7ad0f8946dda24d796c990f89cf: Log is configured to *not* fsync() on all Append() calls
I20260806 22:50:00.522492 32094 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d753d7ad0f8946dda24d796c990f89cf: No bootstrap required, opened a new log
I20260806 22:50:00.522868 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 32026
I20260806 22:50:00.522938 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056586788339-29547-0/minicluster-data/master-0/wal/instance
I20260806 22:50:00.523171 29547 external_mini_cluster.cc:1428] Setting key ebef9081f9caf0ad488d974600ebec8d
I20260806 22:50:00.524472 32094 raft_consensus.cc:348] T 00000000000000000000000000000000 P d753d7ad0f8946dda24d796c990f89cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d753d7ad0f8946dda24d796c990f89cf" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 43941 } }
I20260806 22:50:00.524636 32094 raft_consensus.cc:374] T 00000000000000000000000000000000 P d753d7ad0f8946dda24d796c990f89cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:50:00.524678 32094 raft_consensus.cc:729] T 00000000000000000000000000000000 P d753d7ad0f8946dda24d796c990f89cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d753d7ad0f8946dda24d796c990f89cf, State: Initialized, Role: FOLLOWER
I20260806 22:50:00.524813 32094 consensus_queue.cc:260] T 00000000000000000000000000000000 P d753d7ad0f8946dda24d796c990f89cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d753d7ad0f8946dda24d796c990f89cf" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 43941 } }
I20260806 22:50:00.524900 32094 raft_consensus.cc:388] T 00000000000000000000000000000000 P d753d7ad0f8946dda24d796c990f89cf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:50:00.524952 32094 raft_consensus.cc:482] T 00000000000000000000000000000000 P d753d7ad0f8946dda24d796c990f89cf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:50:00.525007 32094 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d753d7ad0f8946dda24d796c990f89cf [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:50:00.525972 32094 raft_consensus.cc:504] T 00000000000000000000000000000000 P d753d7ad0f8946dda24d796c990f89cf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d753d7ad0f8946dda24d796c990f89cf" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 43941 } }
I20260806 22:50:00.526093 32094 leader_election.cc:302] T 00000000000000000000000000000000 P d753d7ad0f8946dda24d796c990f89cf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d753d7ad0f8946dda24d796c990f89cf; no voters:
I20260806 22:50:00.526283 32094 leader_election.cc:288] T 00000000000000000000000000000000 P d753d7ad0f8946dda24d796c990f89cf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:50:00.526309 32099 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d753d7ad0f8946dda24d796c990f89cf [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:50:00.526425 32099 raft_consensus.cc:686] T 00000000000000000000000000000000 P d753d7ad0f8946dda24d796c990f89cf [term 1 LEADER]: Becoming Leader. State: Replica: d753d7ad0f8946dda24d796c990f89cf, State: Running, Role: LEADER
I20260806 22:50:00.526556 32094 sys_catalog.cc:564] T 00000000000000000000000000000000 P d753d7ad0f8946dda24d796c990f89cf [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:50:00.526573 32099 consensus_queue.cc:237] T 00000000000000000000000000000000 P d753d7ad0f8946dda24d796c990f89cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d753d7ad0f8946dda24d796c990f89cf" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 43941 } }
I20260806 22:50:00.527021 32100 sys_catalog.cc:455] T 00000000000000000000000000000000 P d753d7ad0f8946dda24d796c990f89cf [sys.catalog]: SysCatalogTable state changed. Reason: New leader d753d7ad0f8946dda24d796c990f89cf. Latest consensus state: current_term: 1 leader_uuid: "d753d7ad0f8946dda24d796c990f89cf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d753d7ad0f8946dda24d796c990f89cf" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 43941 } } }
I20260806 22:50:00.527113 32100 sys_catalog.cc:458] T 00000000000000000000000000000000 P d753d7ad0f8946dda24d796c990f89cf [sys.catalog]: This master's current role is: LEADER
I20260806 22:50:00.527377 32101 sys_catalog.cc:455] T 00000000000000000000000000000000 P d753d7ad0f8946dda24d796c990f89cf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d753d7ad0f8946dda24d796c990f89cf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d753d7ad0f8946dda24d796c990f89cf" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 43941 } } }
I20260806 22:50:00.527509 32101 sys_catalog.cc:458] T 00000000000000000000000000000000 P d753d7ad0f8946dda24d796c990f89cf [sys.catalog]: This master's current role is: LEADER
I20260806 22:50:00.527680 32106 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:50:00.528247 32106 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:50:00.530025 32106 catalog_manager.cc:1347] Generated new cluster ID: c8907fcfdc604dedb07a2f90a7b642db
I20260806 22:50:00.530076 32106 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 22:50:00 dist-test-slave-1xrh krb5kdc[32011](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056600, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.218.254@KRBTEST.COM
I20260806 22:50:00.534602 32098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:00.523677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37574 (local address 127.28.218.254:43941)
0806 22:50:00.523863 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:00.523870 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:00.523897 (+ 27us) server_negotiation.cc:407] Connection header received
0806 22:50:00.524453 (+ 556us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:00.524458 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:00.524503 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:00.524608 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:00.525409 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:00.526423 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:00.527545 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:00.528172 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:00.528275 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:00.531013 (+ 2738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:50:00.531016 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:50:00.531017 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:50:00.531027 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:50:00.532998 (+ 1971us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:00.533389 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:00.533392 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:00.533392 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:00.533523 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:00.533722 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:00.533722 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:00.533723 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:00.534045 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:50:00.534211 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:00.534274 (+ 63us) server_negotiation.cc:299] Negotiation successful
0806 22:50:00.534411 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":86,"threads_started":1}
I20260806 22:50:00.548272 32106 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:50:00.549031 32106 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:50:00.553536 32106 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d753d7ad0f8946dda24d796c990f89cf: Generated new TSK 0
I20260806 22:50:00.553714 32106 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.28.218.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.218.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.218.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.193.keytab.
Entry for principal kudu/127.28.218.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.193.keytab.
I20260806 22:50:00.582006 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056586788339-29547-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056586788339-29547-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056586788339-29547-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056586788339-29547-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.218.193:0
--local_ip_for_outbound_sockets=127.28.218.193
--webserver_interface=127.28.218.193
--webserver_port=0
--tserver_master_addrs=127.28.218.254:43941
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.193.keytab
--principal=kudu/127.28.218.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.218.212:39411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:50:00.662977 32122 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:50:00.663185 32122 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:50:00.663208 32122 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:50:00.664309 32122 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:50:00.664345 32122 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:50:00.664371 32122 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.218.193
I20260806 22:50:00.665702 32122 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:39411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056586788339-29547-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056586788339-29547-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.193.keytab
--principal=kudu/127.28.218.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.218.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056586788339-29547-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.218.193
--webserver_port=0
--tserver_master_addrs=127.28.218.254:43941
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32122
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.218.193
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056586788339-29547-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:50:00.665881 32122 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:50:00.666107 32122 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:50:00.670980 32128 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:00.670982 32130 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:50:00.671068 32122 server_base.cc:1034] running on GCE node
W20260806 22:50:00.671042 32127 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:50:00.671444 32122 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:50:00.671639 32122 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:50:00.672794 32122 hybrid_clock.cc:648] HybridClock initialized: now 1786056600672755 us; error 38 us; skew 500 ppm
Aug 06 22:50:00 dist-test-slave-1xrh krb5kdc[32011](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056600, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:50:00.675278 32122 init.cc:377] Logged in from keytab as kudu/127.28.218.193@KRBTEST.COM (short username kudu)
I20260806 22:50:00.675638 32122 webserver.cc:460] Webserver started at http://127.28.218.193:39179/ using document root <none> and password file <none>
I20260806 22:50:00.675804 32122 fs_manager.cc:359] Metadata directory not provided
I20260806 22:50:00.675843 32122 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:50:00.675931 32122 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:50:00.676839 32122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056586788339-29547-0/minicluster-data/ts-0/data/instance:
uuid: "eb052675d1ed41bf9b991c0c4a1bccf9"
format_stamp: "Formatted at 2026-08-06 22:50:00 on dist-test-slave-1xrh"
server_key: "d89f65bbdb871c657569f40cbd9db934"
server_key_iv: "dfee44805651af22ac2226dbacc3e143"
server_key_version: "encryptionkey@0"
I20260806 22:50:00.677129 32122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056586788339-29547-0/minicluster-data/ts-0/wal/instance:
uuid: "eb052675d1ed41bf9b991c0c4a1bccf9"
format_stamp: "Formatted at 2026-08-06 22:50:00 on dist-test-slave-1xrh"
server_key: "d89f65bbdb871c657569f40cbd9db934"
server_key_iv: "dfee44805651af22ac2226dbacc3e143"
server_key_version: "encryptionkey@0"
I20260806 22:50:00.678345 32122 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 22:50:00.678961 32138 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:00.679193 32122 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:50:00.679282 32122 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056586788339-29547-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056586788339-29547-0/minicluster-data/ts-0/wal
uuid: "eb052675d1ed41bf9b991c0c4a1bccf9"
format_stamp: "Formatted at 2026-08-06 22:50:00 on dist-test-slave-1xrh"
server_key: "d89f65bbdb871c657569f40cbd9db934"
server_key_iv: "dfee44805651af22ac2226dbacc3e143"
server_key_version: "encryptionkey@0"
I20260806 22:50:00.679358 32122 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056586788339-29547-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056586788339-29547-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056586788339-29547-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:50:00.686905 32122 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:50:00.688601 32122 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:50:00.688751 32122 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:50:00.688952 32122 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:50:00.689231 32122 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:50:00.689283 32122 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:00.689322 32122 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:50:00.689352 32122 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:00.694866 32122 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.193:46775
I20260806 22:50:00.694952 32262 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.193:46775 every 8 connection(s)
I20260806 22:50:00.695281 32122 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056586788339-29547-0/minicluster-data/ts-0/data/info.pb
I20260806 22:50:00.696431 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 32122
I20260806 22:50:00.696513 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056586788339-29547-0/minicluster-data/ts-0/wal/instance
I20260806 22:50:00.696722 29547 external_mini_cluster.cc:1428] Setting key f2b54f91f1ad364f5f43de2697b7931e
Aug 06 22:50:00 dist-test-slave-1xrh krb5kdc[32011](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056600, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.193@KRBTEST.COM for kudu/127.28.218.254@KRBTEST.COM
I20260806 22:50:00.705399 32098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:00.695932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.193:45697 (local address 127.28.218.254:43941)
0806 22:50:00.695975 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:00.695978 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:00.696190 (+ 212us) server_negotiation.cc:407] Connection header received
0806 22:50:00.696704 (+ 514us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:00.696706 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:00.696722 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:00.696793 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:00.698023 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:00.698538 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:00.699815 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:00.700056 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:00.700177 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:00.702489 (+ 2312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:50:00.702491 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:50:00.702491 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:50:00.702502 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:50:00.704004 (+ 1502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:00.704405 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:00.704412 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:00.704413 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:00.704469 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:00.704750 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:00.704751 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:00.704752 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:00.704866 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:50:00.704944 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:00.705216 (+ 272us) server_negotiation.cc:299] Negotiation successful
0806 22:50:00.705309 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 22:50:00.705544 32266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:00.695901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:43941 (local address 127.28.218.193:45697)
0806 22:50:00.696066 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:00.696105 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:00.696610 (+ 505us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:00.696928 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:00.696932 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:00.697672 (+ 740us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:00.697928 (+ 256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:00.697934 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:00.698629 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:00.698629 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:00.699722 (+ 1093us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:00.699725 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:00.700185 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:00.700186 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:00.700326 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:00.700801 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:50:00.700811 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:00.702392 (+ 1581us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:50:00.704089 (+ 1697us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:00.704096 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:00.704098 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:00.704313 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:00.704577 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:00.704578 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:00.704578 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:00.704680 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:00.704937 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:50:00.704943 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:50:00.705079 (+ 136us) client_negotiation.cc:769] Sending connection context
0806 22:50:00.705180 (+ 101us) client_negotiation.cc:240] Negotiation successful
0806 22:50:00.705328 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":100,"threads_started":1}
I20260806 22:50:00.705987 32264 heartbeater.cc:347] Connected to a master server at 127.28.218.254:43941
WARNING: no policy specified for kudu/127.28.218.194@KRBTEST.COM; defaulting to no policy
I20260806 22:50:00.706099 32264 heartbeater.cc:464] Registering TS with master...
I20260806 22:50:00.706286 32264 heartbeater.cc:511] Master 127.28.218.254:43941 requested a full tablet report, sending...
I20260806 22:50:00.706785 32058 ts_manager.cc:194] Registered new tserver with Master: eb052675d1ed41bf9b991c0c4a1bccf9 (127.28.218.193:46775)
I20260806 22:50:00.707504 32058 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.218.193@KRBTEST.COM'} at 127.28.218.193:45697
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.218.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.218.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.194.keytab.
Entry for principal kudu/127.28.218.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.194.keytab.
I20260806 22:50:00.722831 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056586788339-29547-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056586788339-29547-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056586788339-29547-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056586788339-29547-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.218.194:0
--local_ip_for_outbound_sockets=127.28.218.194
--webserver_interface=127.28.218.194
--webserver_port=0
--tserver_master_addrs=127.28.218.254:43941
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.194.keytab
--principal=kudu/127.28.218.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.218.212:39411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:50:00.802723 32269 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:50:00.802906 32269 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:50:00.802928 32269 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:50:00.804200 32269 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:50:00.804245 32269 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:50:00.804296 32269 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.218.194
I20260806 22:50:00.805670 32269 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:39411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056586788339-29547-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056586788339-29547-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.194.keytab
--principal=kudu/127.28.218.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.218.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056586788339-29547-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.218.194
--webserver_port=0
--tserver_master_addrs=127.28.218.254:43941
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32269
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.218.194
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056586788339-29547-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:50:00.805900 32269 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:50:00.806130 32269 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 22:50:00.811193 32269 server_base.cc:1034] running on GCE node
W20260806 22:50:00.811134 32275 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:00.811080 32274 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:00.811080 32277 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:50:00.811628 32269 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:50:00.811878 32269 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:50:00.813040 32269 hybrid_clock.cc:648] HybridClock initialized: now 1786056600813023 us; error 30 us; skew 500 ppm
Aug 06 22:50:00 dist-test-slave-1xrh krb5kdc[32011](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056600, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:50:00.815660 32269 init.cc:377] Logged in from keytab as kudu/127.28.218.194@KRBTEST.COM (short username kudu)
I20260806 22:50:00.816133 32269 webserver.cc:460] Webserver started at http://127.28.218.194:42215/ using document root <none> and password file <none>
I20260806 22:50:00.816375 32269 fs_manager.cc:359] Metadata directory not provided
I20260806 22:50:00.816442 32269 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:50:00.816576 32269 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:50:00.817557 32269 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056586788339-29547-0/minicluster-data/ts-1/data/instance:
uuid: "144e6e1cfa3e4f88a79cef0beee4d974"
format_stamp: "Formatted at 2026-08-06 22:50:00 on dist-test-slave-1xrh"
server_key: "a5bfccc3dd61a50c20fdf1d508fdbafa"
server_key_iv: "ccdb103e087c93f79b72aaee440a5936"
server_key_version: "encryptionkey@0"
I20260806 22:50:00.817910 32269 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056586788339-29547-0/minicluster-data/ts-1/wal/instance:
uuid: "144e6e1cfa3e4f88a79cef0beee4d974"
format_stamp: "Formatted at 2026-08-06 22:50:00 on dist-test-slave-1xrh"
server_key: "a5bfccc3dd61a50c20fdf1d508fdbafa"
server_key_iv: "ccdb103e087c93f79b72aaee440a5936"
server_key_version: "encryptionkey@0"
I20260806 22:50:00.819209 32269 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 22:50:00.819942 32287 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:00.820106 32269 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:50:00.820212 32269 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056586788339-29547-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056586788339-29547-0/minicluster-data/ts-1/wal
uuid: "144e6e1cfa3e4f88a79cef0beee4d974"
format_stamp: "Formatted at 2026-08-06 22:50:00 on dist-test-slave-1xrh"
server_key: "a5bfccc3dd61a50c20fdf1d508fdbafa"
server_key_iv: "ccdb103e087c93f79b72aaee440a5936"
server_key_version: "encryptionkey@0"
I20260806 22:50:00.820299 32269 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056586788339-29547-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056586788339-29547-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056586788339-29547-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:50:00.838593 32269 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:50:00.840857 32269 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:50:00.841020 32269 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:50:00.841252 32269 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:50:00.841542 32269 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:50:00.841595 32269 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:00.841637 32269 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:50:00.841666 32269 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:00.847293 32269 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.194:35881
I20260806 22:50:00.847348 32412 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.194:35881 every 8 connection(s)
I20260806 22:50:00.847669 32269 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056586788339-29547-0/minicluster-data/ts-1/data/info.pb
Aug 06 22:50:00 dist-test-slave-1xrh krb5kdc[32011](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056600, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.194@KRBTEST.COM for kudu/127.28.218.254@KRBTEST.COM
I20260806 22:50:00.856243 32098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:00.848383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.194:37621 (local address 127.28.218.254:43941)
0806 22:50:00.848469 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:00.848471 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:00.848640 (+ 169us) server_negotiation.cc:407] Connection header received
0806 22:50:00.849186 (+ 546us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:00.849188 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:00.849202 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:00.849253 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:00.850128 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:00.850632 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:00.851702 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:00.851920 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:00.852014 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:00.853901 (+ 1887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:50:00.853902 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:50:00.853902 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:50:00.853915 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:50:00.855087 (+ 1172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:00.855501 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:00.855502 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:00.855502 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:00.855554 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:00.855751 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:00.855751 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:00.855751 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:00.855846 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:50:00.855894 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:00.856116 (+ 222us) server_negotiation.cc:299] Negotiation successful
0806 22:50:00.856181 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 22:50:00.856396 32415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:00.848340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:43941 (local address 127.28.218.194:37621)
0806 22:50:00.848516 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:00.848562 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:00.849082 (+ 520us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:00.849313 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:00.849317 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:00.849853 (+ 536us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:00.850027 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:00.850031 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:00.850733 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:00.850734 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:00.851612 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:00.851615 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:00.852023 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:00.852025 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:00.852143 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:00.852529 (+ 386us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:50:00.852537 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:00.853803 (+ 1266us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:50:00.855230 (+ 1427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:00.855235 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:00.855236 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:00.855436 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:00.855610 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:00.855611 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:00.855611 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:00.855699 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:00.855899 (+ 200us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:50:00.855903 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:50:00.856011 (+ 108us) client_negotiation.cc:769] Sending connection context
0806 22:50:00.856088 (+ 77us) client_negotiation.cc:240] Negotiation successful
0806 22:50:00.856194 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":87,"threads_started":1}
I20260806 22:50:00.856878 32413 heartbeater.cc:347] Connected to a master server at 127.28.218.254:43941
I20260806 22:50:00.856969 32413 heartbeater.cc:464] Registering TS with master...
I20260806 22:50:00.857110 32413 heartbeater.cc:511] Master 127.28.218.254:43941 requested a full tablet report, sending...
I20260806 22:50:00.857435 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 32269
I20260806 22:50:00.857511 32058 ts_manager.cc:194] Registered new tserver with Master: 144e6e1cfa3e4f88a79cef0beee4d974 (127.28.218.194:35881)
I20260806 22:50:00.857553 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056586788339-29547-0/minicluster-data/ts-1/wal/instance
I20260806 22:50:00.857714 29547 external_mini_cluster.cc:1428] Setting key 8f95e6e9f74b8f260ad7dbff22d790d0
I20260806 22:50:00.857971 32058 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.218.194@KRBTEST.COM'} at 127.28.218.194:37621
WARNING: no policy specified for kudu/127.28.218.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.218.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.218.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.195.keytab.
Entry for principal kudu/127.28.218.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.195.keytab.
I20260806 22:50:00.882431 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056586788339-29547-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056586788339-29547-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056586788339-29547-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056586788339-29547-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.218.195:0
--local_ip_for_outbound_sockets=127.28.218.195
--webserver_interface=127.28.218.195
--webserver_port=0
--tserver_master_addrs=127.28.218.254:43941
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.195.keytab
--principal=kudu/127.28.218.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.218.212:39411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:50:00.963729 32418 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:50:00.963963 32418 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:50:00.964003 32418 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:50:00.965150 32418 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:50:00.965219 32418 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:50:00.965263 32418 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.218.195
I20260806 22:50:00.966656 32418 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:39411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056586788339-29547-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056586788339-29547-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.195.keytab
--principal=kudu/127.28.218.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.218.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056586788339-29547-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.218.195
--webserver_port=0
--tserver_master_addrs=127.28.218.254:43941
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32418
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.218.195
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056586788339-29547-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:50:00.966914 32418 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:50:00.967157 32418 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:50:00.972007 32427 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:00.971998 32425 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:00.971987 32424 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:50:00.972048 32418 server_base.cc:1034] running on GCE node
I20260806 22:50:00.972477 32418 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:50:00.972702 32418 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:50:00.973872 32418 hybrid_clock.cc:648] HybridClock initialized: now 1786056600973859 us; error 27 us; skew 500 ppm
Aug 06 22:50:00 dist-test-slave-1xrh krb5kdc[32011](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056600, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:50:00.976405 32418 init.cc:377] Logged in from keytab as kudu/127.28.218.195@KRBTEST.COM (short username kudu)
I20260806 22:50:00.976850 32418 webserver.cc:460] Webserver started at http://127.28.218.195:45471/ using document root <none> and password file <none>
I20260806 22:50:00.977029 32418 fs_manager.cc:359] Metadata directory not provided
I20260806 22:50:00.977068 32418 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:50:00.977154 32418 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:50:00.978091 32418 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056586788339-29547-0/minicluster-data/ts-2/data/instance:
uuid: "9e7209fa2aa64fbcac85c302783a2664"
format_stamp: "Formatted at 2026-08-06 22:50:00 on dist-test-slave-1xrh"
server_key: "920ac845f942b06eb21493fc269a3e17"
server_key_iv: "e97d73e7430275c9048e9f5a7679b8be"
server_key_version: "encryptionkey@0"
I20260806 22:50:00.978376 32418 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056586788339-29547-0/minicluster-data/ts-2/wal/instance:
uuid: "9e7209fa2aa64fbcac85c302783a2664"
format_stamp: "Formatted at 2026-08-06 22:50:00 on dist-test-slave-1xrh"
server_key: "920ac845f942b06eb21493fc269a3e17"
server_key_iv: "e97d73e7430275c9048e9f5a7679b8be"
server_key_version: "encryptionkey@0"
I20260806 22:50:00.979679 32418 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:50:00.980346 32436 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:00.980538 32418 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:50:00.980633 32418 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056586788339-29547-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056586788339-29547-0/minicluster-data/ts-2/wal
uuid: "9e7209fa2aa64fbcac85c302783a2664"
format_stamp: "Formatted at 2026-08-06 22:50:00 on dist-test-slave-1xrh"
server_key: "920ac845f942b06eb21493fc269a3e17"
server_key_iv: "e97d73e7430275c9048e9f5a7679b8be"
server_key_version: "encryptionkey@0"
I20260806 22:50:00.980684 32418 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056586788339-29547-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056586788339-29547-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056586788339-29547-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:50:01.001992 32418 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:50:01.004035 32418 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:50:01.004227 32418 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:50:01.004419 32418 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:50:01.004722 32418 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:50:01.004756 32418 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:01.004779 32418 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:50:01.004827 32418 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:01.010671 32418 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.195:43341
I20260806 22:50:01.010746 32559 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.195:43341 every 8 connection(s)
I20260806 22:50:01.011050 32418 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056586788339-29547-0/minicluster-data/ts-2/data/info.pb
Aug 06 22:50:01 dist-test-slave-1xrh krb5kdc[32011](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056600, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.195@KRBTEST.COM for kudu/127.28.218.254@KRBTEST.COM
I20260806 22:50:01.016883 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 32418
I20260806 22:50:01.017025 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056586788339-29547-0/minicluster-data/ts-2/wal/instance
I20260806 22:50:01.017212 29547 external_mini_cluster.cc:1428] Setting key b820e26fd3689a44983eb9d60cb0143d
I20260806 22:50:01.020359 32098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:01.011783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.195:36375 (local address 127.28.218.254:43941)
0806 22:50:01.011861 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:01.011863 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:01.012064 (+ 201us) server_negotiation.cc:407] Connection header received
0806 22:50:01.012507 (+ 443us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:01.012508 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:01.012518 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:01.012568 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:01.013463 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:01.013943 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:01.014968 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:01.015223 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:01.015280 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:01.017442 (+ 2162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:50:01.017444 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:50:01.017444 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:50:01.017451 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:50:01.018964 (+ 1513us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:01.019427 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:01.019428 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:01.019429 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:01.019491 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:01.019738 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:01.019739 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:01.019739 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:01.019851 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:50:01.019925 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:01.020190 (+ 265us) server_negotiation.cc:299] Negotiation successful
0806 22:50:01.020271 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 22:50:01.020537 32562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:01.011753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:43941 (local address 127.28.218.195:36375)
0806 22:50:01.011936 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:01.011979 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:01.012425 (+ 446us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:01.012619 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:01.012622 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:01.013199 (+ 577us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:01.013387 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:01.013392 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:01.014051 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:01.014052 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:01.014874 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:01.014876 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:01.015296 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:01.015297 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:01.015397 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:01.015822 (+ 425us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:50:01.015831 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:01.017345 (+ 1514us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:50:01.019066 (+ 1721us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:01.019073 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:01.019075 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:01.019346 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:01.019557 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:01.019558 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:01.019558 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:01.019670 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:01.019922 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:50:01.019929 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:50:01.020058 (+ 129us) client_negotiation.cc:769] Sending connection context
0806 22:50:01.020167 (+ 109us) client_negotiation.cc:240] Negotiation successful
0806 22:50:01.020317 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":106,"threads_started":1}
I20260806 22:50:01.020946 32560 heartbeater.cc:347] Connected to a master server at 127.28.218.254:43941
I20260806 22:50:01.021049 32560 heartbeater.cc:464] Registering TS with master...
I20260806 22:50:01.021212 32560 heartbeater.cc:511] Master 127.28.218.254:43941 requested a full tablet report, sending...
I20260806 22:50:01.021610 32057 ts_manager.cc:194] Registered new tserver with Master: 9e7209fa2aa64fbcac85c302783a2664 (127.28.218.195:43341)
I20260806 22:50:01.021994 32057 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.218.195@KRBTEST.COM'} at 127.28.218.195:36375
I20260806 22:50:01.028885 29547 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 22:50:01.040436 32098 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:50:01.039654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37578 (local address 127.28.218.254:43941)
0806 22:50:01.039699 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:01.039701 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:01.039750 (+ 49us) server_negotiation.cc:407] Connection header received
0806 22:50:01.039802 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:01.039803 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:01.039812 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:01.039856 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:01.040333 (+ 477us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37578: BlockingRecv error: recv got EOF from 127.0.0.1:37578 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 22:50:01.041188 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 32122
I20260806 22:50:01.046921 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 32269
I20260806 22:50:01.052363 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 32418
I20260806 22:50:01.057169 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 32026
2026-08-06T22:50:01Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (1840 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786056586788339-29547-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:50:01 dist-test-slave-1xrh krb5kdc[32573](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:50:01 dist-test-slave-1xrh krb5kdc[32573](info): set up 2 sockets
Aug 06 22:50:01 dist-test-slave-1xrh krb5kdc[32573](info): commencing operation
krb5kdc: starting...
W20260806 22:50:02.117794 29547 mini_kdc.cc:121] Time spent starting KDC: real 1.040s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786056586788339-29547-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786056586788339-29547-0/krb5kdc/test-user.keytab.
Aug 06 22:50:02 dist-test-slave-1xrh krb5kdc[32573](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056602, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:50:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:50:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.218.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.218.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.218.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab.
Entry for principal kudu/127.28.218.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab.
I20260806 22:50:02.233788 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786056586788339-29547-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786056586788339-29547-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786056586788339-29547-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.218.254:40801
--webserver_interface=127.28.218.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab
--principal=kudu/127.28.218.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.218.212:41955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.218.254:40801
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:50:02.312871 32586 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:50:02.313097 32586 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:50:02.313136 32586 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:50:02.314253 32586 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:50:02.314316 32586 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:50:02.314343 32586 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:50:02.314374 32586 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:50:02.314399 32586 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:50:02.315920 32586 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:41955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786056586788339-29547-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786056586788339-29547-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.218.254:40801
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab
--principal=kudu/127.28.218.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.218.254:40801
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.218.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32586
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786056586788339-29547-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:50:02.316192 32586 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:50:02.316411 32586 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:50:02.321081 32594 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:02.321123 32592 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:02.321089 32591 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:50:02.321246 32586 server_base.cc:1034] running on GCE node
I20260806 22:50:02.321593 32586 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:50:02.322062 32586 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:50:02.323223 32586 hybrid_clock.cc:648] HybridClock initialized: now 1786056602323190 us; error 36 us; skew 500 ppm
Aug 06 22:50:02 dist-test-slave-1xrh krb5kdc[32573](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056602, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:50:02.325757 32586 init.cc:377] Logged in from keytab as kudu/127.28.218.254@KRBTEST.COM (short username kudu)
I20260806 22:50:02.326200 32586 webserver.cc:460] Webserver started at http://127.28.218.254:41955/ using document root <none> and password file <none>
I20260806 22:50:02.326471 32586 fs_manager.cc:359] Metadata directory not provided
I20260806 22:50:02.326548 32586 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:50:02.326727 32586 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:50:02.328012 32586 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786056586788339-29547-0/minicluster-data/master-0/data/instance:
uuid: "87f6489f5b9a49399a6ab0dbf9e425e5"
format_stamp: "Formatted at 2026-08-06 22:50:02 on dist-test-slave-1xrh"
server_key: "aa1814035d35d98060cedabe1f94fdaf"
server_key_iv: "c1649f54cd5f6c7819fdf68211ca321c"
server_key_version: "encryptionkey@0"
I20260806 22:50:02.328401 32586 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786056586788339-29547-0/minicluster-data/master-0/wal/instance:
uuid: "87f6489f5b9a49399a6ab0dbf9e425e5"
format_stamp: "Formatted at 2026-08-06 22:50:02 on dist-test-slave-1xrh"
server_key: "aa1814035d35d98060cedabe1f94fdaf"
server_key_iv: "c1649f54cd5f6c7819fdf68211ca321c"
server_key_version: "encryptionkey@0"
I20260806 22:50:02.329803 32586 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:50:02.330560 32601 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:02.330755 32586 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:50:02.330864 32586 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786056586788339-29547-0/minicluster-data/master-0/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786056586788339-29547-0/minicluster-data/master-0/wal
uuid: "87f6489f5b9a49399a6ab0dbf9e425e5"
format_stamp: "Formatted at 2026-08-06 22:50:02 on dist-test-slave-1xrh"
server_key: "aa1814035d35d98060cedabe1f94fdaf"
server_key_iv: "c1649f54cd5f6c7819fdf68211ca321c"
server_key_version: "encryptionkey@0"
I20260806 22:50:02.330940 32586 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786056586788339-29547-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786056586788339-29547-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786056586788339-29547-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:50:02.348369 32586 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:50:02.350205 32586 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:50:02.350379 32586 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:50:02.354832 32586 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.254:40801
I20260806 22:50:02.354863 32659 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.254:40801 every 8 connection(s)
I20260806 22:50:02.355404 32586 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
I20260806 22:50:02.356137 32660 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:50:02.358039 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 32586
I20260806 22:50:02.358126 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786056586788339-29547-0/minicluster-data/master-0/wal/instance
I20260806 22:50:02.358325 29547 external_mini_cluster.cc:1428] Setting key 80323e29771ff3aa4ae4f09435bed785
I20260806 22:50:02.358420 32660 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87f6489f5b9a49399a6ab0dbf9e425e5: Bootstrap starting.
I20260806 22:50:02.359181 32660 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 87f6489f5b9a49399a6ab0dbf9e425e5: Neither blocks nor log segments found. Creating new log.
I20260806 22:50:02.359519 32660 log.cc:824] T 00000000000000000000000000000000 P 87f6489f5b9a49399a6ab0dbf9e425e5: Log is configured to *not* fsync() on all Append() calls
I20260806 22:50:02.360292 32660 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87f6489f5b9a49399a6ab0dbf9e425e5: No bootstrap required, opened a new log
I20260806 22:50:02.362625 32660 raft_consensus.cc:348] T 00000000000000000000000000000000 P 87f6489f5b9a49399a6ab0dbf9e425e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87f6489f5b9a49399a6ab0dbf9e425e5" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 40801 } }
I20260806 22:50:02.362751 32660 raft_consensus.cc:374] T 00000000000000000000000000000000 P 87f6489f5b9a49399a6ab0dbf9e425e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:50:02.362773 32660 raft_consensus.cc:729] T 00000000000000000000000000000000 P 87f6489f5b9a49399a6ab0dbf9e425e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87f6489f5b9a49399a6ab0dbf9e425e5, State: Initialized, Role: FOLLOWER
I20260806 22:50:02.362922 32660 consensus_queue.cc:260] T 00000000000000000000000000000000 P 87f6489f5b9a49399a6ab0dbf9e425e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87f6489f5b9a49399a6ab0dbf9e425e5" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 40801 } }
I20260806 22:50:02.363008 32660 raft_consensus.cc:388] T 00000000000000000000000000000000 P 87f6489f5b9a49399a6ab0dbf9e425e5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:50:02.363044 32660 raft_consensus.cc:482] T 00000000000000000000000000000000 P 87f6489f5b9a49399a6ab0dbf9e425e5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:50:02.363101 32660 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 87f6489f5b9a49399a6ab0dbf9e425e5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:50:02.363639 32660 raft_consensus.cc:504] T 00000000000000000000000000000000 P 87f6489f5b9a49399a6ab0dbf9e425e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87f6489f5b9a49399a6ab0dbf9e425e5" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 40801 } }
I20260806 22:50:02.363755 32660 leader_election.cc:302] T 00000000000000000000000000000000 P 87f6489f5b9a49399a6ab0dbf9e425e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 87f6489f5b9a49399a6ab0dbf9e425e5; no voters:
I20260806 22:50:02.363931 32660 leader_election.cc:288] T 00000000000000000000000000000000 P 87f6489f5b9a49399a6ab0dbf9e425e5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:50:02.363978 32666 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 87f6489f5b9a49399a6ab0dbf9e425e5 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:50:02.364143 32666 raft_consensus.cc:686] T 00000000000000000000000000000000 P 87f6489f5b9a49399a6ab0dbf9e425e5 [term 1 LEADER]: Becoming Leader. State: Replica: 87f6489f5b9a49399a6ab0dbf9e425e5, State: Running, Role: LEADER
I20260806 22:50:02.364295 32666 consensus_queue.cc:237] T 00000000000000000000000000000000 P 87f6489f5b9a49399a6ab0dbf9e425e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87f6489f5b9a49399a6ab0dbf9e425e5" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 40801 } }
I20260806 22:50:02.364179 32660 sys_catalog.cc:564] T 00000000000000000000000000000000 P 87f6489f5b9a49399a6ab0dbf9e425e5 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:50:02.364712 32667 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87f6489f5b9a49399a6ab0dbf9e425e5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "87f6489f5b9a49399a6ab0dbf9e425e5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87f6489f5b9a49399a6ab0dbf9e425e5" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 40801 } } }
I20260806 22:50:02.364902 32667 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87f6489f5b9a49399a6ab0dbf9e425e5 [sys.catalog]: This master's current role is: LEADER
I20260806 22:50:02.365418 32675 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:50:02.365600 32668 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87f6489f5b9a49399a6ab0dbf9e425e5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 87f6489f5b9a49399a6ab0dbf9e425e5. Latest consensus state: current_term: 1 leader_uuid: "87f6489f5b9a49399a6ab0dbf9e425e5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87f6489f5b9a49399a6ab0dbf9e425e5" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 40801 } } }
I20260806 22:50:02.365754 32668 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87f6489f5b9a49399a6ab0dbf9e425e5 [sys.catalog]: This master's current role is: LEADER
I20260806 22:50:02.365958 32675 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 06 22:50:02 dist-test-slave-1xrh krb5kdc[32573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056602, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.218.254@KRBTEST.COM
I20260806 22:50:02.368012 32675 catalog_manager.cc:1347] Generated new cluster ID: d5ef7f5c8c0d419088b7ceb6aa51eeea
I20260806 22:50:02.368069 32675 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:50:02.370419 32664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:02.358864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59028 (local address 127.28.218.254:40801)
0806 22:50:02.359049 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:02.359058 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:02.359096 (+ 38us) server_negotiation.cc:407] Connection header received
0806 22:50:02.359709 (+ 613us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:02.359718 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:02.359777 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:02.359899 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:02.360696 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:02.362083 (+ 1387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:02.362923 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:02.363464 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:02.363604 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:02.366779 (+ 3175us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:50:02.366784 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:50:02.366784 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:50:02.366797 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:50:02.368772 (+ 1975us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:02.369189 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:02.369192 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:02.369193 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:02.369292 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:02.369527 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:02.369528 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:02.369529 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:02.369779 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:50:02.369949 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:02.370070 (+ 121us) server_negotiation.cc:299] Negotiation successful
0806 22:50:02.370220 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":91,"threads_started":1}
I20260806 22:50:02.372133 29547 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:50:02.383888 32675 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:50:02.384616 32675 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:50:02.388484 32664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:02.384425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59044 (local address 127.28.218.254:40801)
0806 22:50:02.384471 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:02.384473 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:02.384651 (+ 178us) server_negotiation.cc:407] Connection header received
0806 22:50:02.384756 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:02.384758 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:02.384770 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:02.384855 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:02.385261 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:02.385827 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:02.386641 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:02.386871 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:02.386956 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:02.388005 (+ 1049us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:50:02.388007 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:50:02.388007 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:50:02.388176 (+ 169us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:50:02.388192 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:50:02.388198 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:50:02.388279 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:02.388384 (+ 105us) server_negotiation.cc:299] Negotiation successful
0806 22:50:02.388412 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 22:50:02.388711 32675 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 87f6489f5b9a49399a6ab0dbf9e425e5: Generated new TSK 0
I20260806 22:50:02.388902 32675 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:50:02.396333 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 32586
2026-08-06T22:50:02Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1337 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056586788339-29547-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:50:02 dist-test-slave-1xrh krb5kdc[32699](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:50:02 dist-test-slave-1xrh krb5kdc[32699](info): set up 2 sockets
Aug 06 22:50:02 dist-test-slave-1xrh krb5kdc[32699](info): commencing operation
krb5kdc: starting...
W20260806 22:50:03.463198 29547 mini_kdc.cc:121] Time spent starting KDC: real 1.048s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056586788339-29547-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056586788339-29547-0/krb5kdc/test-user.keytab.
Aug 06 22:50:03 dist-test-slave-1xrh krb5kdc[32699](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056603, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:50:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:50:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.218.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.218.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.218.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab.
Entry for principal kudu/127.28.218.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab.
I20260806 22:50:03.577777 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056586788339-29547-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056586788339-29547-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056586788339-29547-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.218.254:35269
--webserver_interface=127.28.218.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab
--principal=kudu/127.28.218.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.218.212:43069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.218.254:35269
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:50:03.658667 32713 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:50:03.658860 32713 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:50:03.658881 32713 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:50:03.659951 32713 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:50:03.659986 32713 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:50:03.660000 32713 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:50:03.660012 32713 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:50:03.660024 32713 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:50:03.661346 32713 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:43069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056586788339-29547-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056586788339-29547-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.218.254:35269
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab
--principal=kudu/127.28.218.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.218.254:35269
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.218.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32713
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056586788339-29547-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:50:03.661532 32713 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:50:03.661684 32713 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:50:03.666353 32720 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:03.666365 32722 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:03.666365 32719 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:50:03.666601 32713 server_base.cc:1034] running on GCE node
I20260806 22:50:03.666842 32713 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:50:03.667078 32713 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:50:03.668212 32713 hybrid_clock.cc:648] HybridClock initialized: now 1786056603668203 us; error 28 us; skew 500 ppm
Aug 06 22:50:03 dist-test-slave-1xrh krb5kdc[32699](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056603, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:50:03.670591 32713 init.cc:377] Logged in from keytab as kudu/127.28.218.254@KRBTEST.COM (short username kudu)
I20260806 22:50:03.671041 32713 webserver.cc:460] Webserver started at http://127.28.218.254:34003/ using document root <none> and password file <none>
I20260806 22:50:03.671299 32713 fs_manager.cc:359] Metadata directory not provided
I20260806 22:50:03.671365 32713 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:50:03.671592 32713 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:50:03.672596 32713 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056586788339-29547-0/minicluster-data/master-0/data/instance:
uuid: "08e166ce427c43e690a3f9ca8f05e0dc"
format_stamp: "Formatted at 2026-08-06 22:50:03 on dist-test-slave-1xrh"
server_key: "290b3a8fc691ead49e8552fadd780c25"
server_key_iv: "f0144f805a0dad6e51f4ef3fdb613168"
server_key_version: "encryptionkey@0"
I20260806 22:50:03.672953 32713 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056586788339-29547-0/minicluster-data/master-0/wal/instance:
uuid: "08e166ce427c43e690a3f9ca8f05e0dc"
format_stamp: "Formatted at 2026-08-06 22:50:03 on dist-test-slave-1xrh"
server_key: "290b3a8fc691ead49e8552fadd780c25"
server_key_iv: "f0144f805a0dad6e51f4ef3fdb613168"
server_key_version: "encryptionkey@0"
I20260806 22:50:03.674237 32713 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:50:03.674935 32730 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:03.675108 32713 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:50:03.675239 32713 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056586788339-29547-0/minicluster-data/master-0/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056586788339-29547-0/minicluster-data/master-0/wal
uuid: "08e166ce427c43e690a3f9ca8f05e0dc"
format_stamp: "Formatted at 2026-08-06 22:50:03 on dist-test-slave-1xrh"
server_key: "290b3a8fc691ead49e8552fadd780c25"
server_key_iv: "f0144f805a0dad6e51f4ef3fdb613168"
server_key_version: "encryptionkey@0"
I20260806 22:50:03.675325 32713 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056586788339-29547-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056586788339-29547-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056586788339-29547-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:50:03.688217 32713 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:50:03.689980 32713 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:50:03.690145 32713 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:50:03.694265 32713 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.254:35269
I20260806 22:50:03.694299 320 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.254:35269 every 8 connection(s)
I20260806 22:50:03.694657 32713 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
I20260806 22:50:03.695765 321 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:50:03.697890 321 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 08e166ce427c43e690a3f9ca8f05e0dc: Bootstrap starting.
I20260806 22:50:03.698398 321 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 08e166ce427c43e690a3f9ca8f05e0dc: Neither blocks nor log segments found. Creating new log.
I20260806 22:50:03.698635 321 log.cc:824] T 00000000000000000000000000000000 P 08e166ce427c43e690a3f9ca8f05e0dc: Log is configured to *not* fsync() on all Append() calls
I20260806 22:50:03.699295 321 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 08e166ce427c43e690a3f9ca8f05e0dc: No bootstrap required, opened a new log
I20260806 22:50:03.700801 321 raft_consensus.cc:348] T 00000000000000000000000000000000 P 08e166ce427c43e690a3f9ca8f05e0dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08e166ce427c43e690a3f9ca8f05e0dc" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 35269 } }
I20260806 22:50:03.700932 321 raft_consensus.cc:374] T 00000000000000000000000000000000 P 08e166ce427c43e690a3f9ca8f05e0dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:50:03.700953 321 raft_consensus.cc:729] T 00000000000000000000000000000000 P 08e166ce427c43e690a3f9ca8f05e0dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08e166ce427c43e690a3f9ca8f05e0dc, State: Initialized, Role: FOLLOWER
I20260806 22:50:03.701084 321 consensus_queue.cc:260] T 00000000000000000000000000000000 P 08e166ce427c43e690a3f9ca8f05e0dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08e166ce427c43e690a3f9ca8f05e0dc" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 35269 } }
I20260806 22:50:03.701162 321 raft_consensus.cc:388] T 00000000000000000000000000000000 P 08e166ce427c43e690a3f9ca8f05e0dc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:50:03.701185 321 raft_consensus.cc:482] T 00000000000000000000000000000000 P 08e166ce427c43e690a3f9ca8f05e0dc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:50:03.701213 321 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 08e166ce427c43e690a3f9ca8f05e0dc [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:50:03.701964 321 raft_consensus.cc:504] T 00000000000000000000000000000000 P 08e166ce427c43e690a3f9ca8f05e0dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08e166ce427c43e690a3f9ca8f05e0dc" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 35269 } }
I20260806 22:50:03.702049 321 leader_election.cc:302] T 00000000000000000000000000000000 P 08e166ce427c43e690a3f9ca8f05e0dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 08e166ce427c43e690a3f9ca8f05e0dc; no voters:
I20260806 22:50:03.702162 321 leader_election.cc:288] T 00000000000000000000000000000000 P 08e166ce427c43e690a3f9ca8f05e0dc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:50:03.702214 326 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 08e166ce427c43e690a3f9ca8f05e0dc [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:50:03.702396 326 raft_consensus.cc:686] T 00000000000000000000000000000000 P 08e166ce427c43e690a3f9ca8f05e0dc [term 1 LEADER]: Becoming Leader. State: Replica: 08e166ce427c43e690a3f9ca8f05e0dc, State: Running, Role: LEADER
I20260806 22:50:03.702474 321 sys_catalog.cc:564] T 00000000000000000000000000000000 P 08e166ce427c43e690a3f9ca8f05e0dc [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:50:03.702531 326 consensus_queue.cc:237] T 00000000000000000000000000000000 P 08e166ce427c43e690a3f9ca8f05e0dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08e166ce427c43e690a3f9ca8f05e0dc" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 35269 } }
I20260806 22:50:03.702598 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 32713
I20260806 22:50:03.702706 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056586788339-29547-0/minicluster-data/master-0/wal/instance
I20260806 22:50:03.702837 326 sys_catalog.cc:455] T 00000000000000000000000000000000 P 08e166ce427c43e690a3f9ca8f05e0dc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 08e166ce427c43e690a3f9ca8f05e0dc. Latest consensus state: current_term: 1 leader_uuid: "08e166ce427c43e690a3f9ca8f05e0dc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08e166ce427c43e690a3f9ca8f05e0dc" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 35269 } } }
I20260806 22:50:03.702893 29547 external_mini_cluster.cc:1428] Setting key 032110a5ecbbc0feb4af78d0f752260f
I20260806 22:50:03.702903 326 sys_catalog.cc:458] T 00000000000000000000000000000000 P 08e166ce427c43e690a3f9ca8f05e0dc [sys.catalog]: This master's current role is: LEADER
I20260806 22:50:03.703130 327 sys_catalog.cc:455] T 00000000000000000000000000000000 P 08e166ce427c43e690a3f9ca8f05e0dc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "08e166ce427c43e690a3f9ca8f05e0dc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08e166ce427c43e690a3f9ca8f05e0dc" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 35269 } } }
I20260806 22:50:03.703231 327 sys_catalog.cc:458] T 00000000000000000000000000000000 P 08e166ce427c43e690a3f9ca8f05e0dc [sys.catalog]: This master's current role is: LEADER
I20260806 22:50:03.703166 330 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:50:03.704205 330 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:50:03.707305 330 catalog_manager.cc:1347] Generated new cluster ID: 83009cfe3c3f41d0b5a681efc3b2fae7
I20260806 22:50:03.707362 330 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 22:50:03 dist-test-slave-1xrh krb5kdc[32699](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056603, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.218.254@KRBTEST.COM
I20260806 22:50:03.713562 330 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:50:03.714190 330 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:50:03.715039 335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:03.703546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55226 (local address 127.28.218.254:35269)
0806 22:50:03.703744 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:03.703752 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:03.703788 (+ 36us) server_negotiation.cc:407] Connection header received
0806 22:50:03.703970 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:03.703977 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:03.704047 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:03.705062 (+ 1015us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:03.705156 (+ 94us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:03.706111 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:03.707157 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:03.707750 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:03.707860 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:03.710571 (+ 2711us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:50:03.710575 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:50:03.710575 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:50:03.710587 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:50:03.713256 (+ 2669us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:03.713716 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:03.713719 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:03.713719 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:03.713830 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:03.714063 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:03.714064 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:03.714064 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:03.714342 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:50:03.714540 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:03.714603 (+ 63us) server_negotiation.cc:299] Negotiation successful
0806 22:50:03.714838 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":99,"threads_started":1}
I20260806 22:50:03.718816 330 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 08e166ce427c43e690a3f9ca8f05e0dc: Generated new TSK 0
I20260806 22:50:03.718981 330 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.28.218.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.218.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.218.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.193.keytab.
Entry for principal kudu/127.28.218.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.193.keytab.
I20260806 22:50:03.748167 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056586788339-29547-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056586788339-29547-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056586788339-29547-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056586788339-29547-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.218.193:0
--local_ip_for_outbound_sockets=127.28.218.193
--webserver_interface=127.28.218.193
--webserver_port=0
--tserver_master_addrs=127.28.218.254:35269
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.193.keytab
--principal=kudu/127.28.218.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.218.212:43069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:50:03.828361 350 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:50:03.828578 350 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:50:03.828619 350 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:50:03.829739 350 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:50:03.829806 350 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:50:03.829851 350 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.218.193
I20260806 22:50:03.831349 350 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:43069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056586788339-29547-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056586788339-29547-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.193.keytab
--principal=kudu/127.28.218.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.218.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056586788339-29547-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.218.193
--webserver_port=0
--tserver_master_addrs=127.28.218.254:35269
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.350
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.218.193
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056586788339-29547-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:50:03.831590 350 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:50:03.831810 350 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:50:03.836637 360 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:03.836659 358 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:03.836606 357 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:50:03.836752 350 server_base.cc:1034] running on GCE node
I20260806 22:50:03.836995 350 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:50:03.837234 350 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:50:03.838394 350 hybrid_clock.cc:648] HybridClock initialized: now 1786056603838380 us; error 43 us; skew 500 ppm
Aug 06 22:50:03 dist-test-slave-1xrh krb5kdc[32699](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056603, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:50:03.840824 350 init.cc:377] Logged in from keytab as kudu/127.28.218.193@KRBTEST.COM (short username kudu)
I20260806 22:50:03.841233 350 webserver.cc:460] Webserver started at http://127.28.218.193:38921/ using document root <none> and password file <none>
I20260806 22:50:03.841475 350 fs_manager.cc:359] Metadata directory not provided
I20260806 22:50:03.841545 350 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:50:03.841689 350 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:50:03.842687 350 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056586788339-29547-0/minicluster-data/ts-0/data/instance:
uuid: "26f08d93a4134e0b8b9162fd198035bd"
format_stamp: "Formatted at 2026-08-06 22:50:03 on dist-test-slave-1xrh"
server_key: "e651ef695931f83617979793c968bb97"
server_key_iv: "512186fe156511c97e648dedbf07b890"
server_key_version: "encryptionkey@0"
I20260806 22:50:03.843042 350 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056586788339-29547-0/minicluster-data/ts-0/wal/instance:
uuid: "26f08d93a4134e0b8b9162fd198035bd"
format_stamp: "Formatted at 2026-08-06 22:50:03 on dist-test-slave-1xrh"
server_key: "e651ef695931f83617979793c968bb97"
server_key_iv: "512186fe156511c97e648dedbf07b890"
server_key_version: "encryptionkey@0"
I20260806 22:50:03.844350 350 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:50:03.845028 367 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:03.845194 350 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:50:03.845293 350 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056586788339-29547-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056586788339-29547-0/minicluster-data/ts-0/wal
uuid: "26f08d93a4134e0b8b9162fd198035bd"
format_stamp: "Formatted at 2026-08-06 22:50:03 on dist-test-slave-1xrh"
server_key: "e651ef695931f83617979793c968bb97"
server_key_iv: "512186fe156511c97e648dedbf07b890"
server_key_version: "encryptionkey@0"
I20260806 22:50:03.845369 350 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056586788339-29547-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056586788339-29547-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056586788339-29547-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:50:03.854105 350 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:50:03.855996 350 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:50:03.856153 350 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:50:03.856374 350 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:50:03.856665 350 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:50:03.856720 350 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:03.856760 350 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:50:03.856788 350 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:03.862339 350 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.193:35803
I20260806 22:50:03.862372 488 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.193:35803 every 8 connection(s)
I20260806 22:50:03.862695 350 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056586788339-29547-0/minicluster-data/ts-0/data/info.pb
Aug 06 22:50:03 dist-test-slave-1xrh krb5kdc[32699](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056603, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.193@KRBTEST.COM for kudu/127.28.218.254@KRBTEST.COM
I20260806 22:50:03.872298 335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:03.863465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.193:58881 (local address 127.28.218.254:35269)
0806 22:50:03.863507 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:03.863510 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:03.863702 (+ 192us) server_negotiation.cc:407] Connection header received
0806 22:50:03.864253 (+ 551us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:03.864255 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:03.864269 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:03.864342 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:03.865271 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:03.865785 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:03.866802 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:03.867018 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:03.867093 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:03.869142 (+ 2049us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:50:03.869143 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:50:03.869144 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:50:03.869151 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:50:03.870408 (+ 1257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:03.870966 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:03.870967 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:03.870967 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:03.871052 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:03.871432 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:03.871433 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:03.871433 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:03.871623 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:50:03.871727 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:03.872110 (+ 383us) server_negotiation.cc:299] Negotiation successful
0806 22:50:03.872197 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 22:50:03.872614 491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:03.863422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:35269 (local address 127.28.218.193:58881)
0806 22:50:03.863572 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:03.863622 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:03.864133 (+ 511us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:03.864408 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:03.864411 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:03.864983 (+ 572us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:03.865170 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:03.865174 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:03.865890 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:03.865892 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:03.866694 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:03.866696 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:03.867113 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:03.867115 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:03.867261 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:03.867734 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:50:03.867742 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:03.869045 (+ 1303us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:50:03.870523 (+ 1478us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:03.870530 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:03.870532 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:03.870850 (+ 318us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:03.871167 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:03.871168 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:03.871173 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:03.871320 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:03.871728 (+ 408us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:50:03.871736 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:50:03.871896 (+ 160us) client_negotiation.cc:769] Sending connection context
0806 22:50:03.872090 (+ 194us) client_negotiation.cc:240] Negotiation successful
0806 22:50:03.872315 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":75,"threads_started":1}
I20260806 22:50:03.872977 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 350
I20260806 22:50:03.873116 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056586788339-29547-0/minicluster-data/ts-0/wal/instance
I20260806 22:50:03.873363 29547 external_mini_cluster.cc:1428] Setting key cc7bc543731bd21c3dbdbdb9e34291bd
I20260806 22:50:03.873322 489 heartbeater.cc:347] Connected to a master server at 127.28.218.254:35269
I20260806 22:50:03.873471 489 heartbeater.cc:464] Registering TS with master...
I20260806 22:50:03.873788 489 heartbeater.cc:511] Master 127.28.218.254:35269 requested a full tablet report, sending...
I20260806 22:50:03.874485 32748 ts_manager.cc:194] Registered new tserver with Master: 26f08d93a4134e0b8b9162fd198035bd (127.28.218.193:35803)
I20260806 22:50:03.875372 32748 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.218.193@KRBTEST.COM'} at 127.28.218.193:58881
WARNING: no policy specified for kudu/127.28.218.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.218.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.218.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.194.keytab.
Entry for principal kudu/127.28.218.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.194.keytab.
I20260806 22:50:03.898355 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056586788339-29547-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056586788339-29547-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056586788339-29547-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056586788339-29547-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.218.194:0
--local_ip_for_outbound_sockets=127.28.218.194
--webserver_interface=127.28.218.194
--webserver_port=0
--tserver_master_addrs=127.28.218.254:35269
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.194.keytab
--principal=kudu/127.28.218.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.218.212:43069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:50:03.979372 495 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:50:03.979573 495 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:50:03.979597 495 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:50:03.980675 495 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:50:03.980712 495 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:50:03.980737 495 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.218.194
I20260806 22:50:03.982115 495 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:43069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056586788339-29547-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056586788339-29547-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.194.keytab
--principal=kudu/127.28.218.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.218.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056586788339-29547-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.218.194
--webserver_port=0
--tserver_master_addrs=127.28.218.254:35269
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.495
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.218.194
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056586788339-29547-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:50:03.982302 495 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:50:03.982468 495 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:50:03.987452 500 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:50:03.987594 495 server_base.cc:1034] running on GCE node
W20260806 22:50:03.987423 504 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:03.987430 501 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:50:03.987968 495 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:50:03.988212 495 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:50:03.989357 495 hybrid_clock.cc:648] HybridClock initialized: now 1786056603989358 us; error 40 us; skew 500 ppm
Aug 06 22:50:03 dist-test-slave-1xrh krb5kdc[32699](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056603, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:50:03.991868 495 init.cc:377] Logged in from keytab as kudu/127.28.218.194@KRBTEST.COM (short username kudu)
I20260806 22:50:03.992331 495 webserver.cc:460] Webserver started at http://127.28.218.194:38691/ using document root <none> and password file <none>
I20260806 22:50:03.992514 495 fs_manager.cc:359] Metadata directory not provided
I20260806 22:50:03.992554 495 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:50:03.992645 495 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:50:03.993633 495 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056586788339-29547-0/minicluster-data/ts-1/data/instance:
uuid: "72dea20d69284cd9a70d4a2e0ebfd0ce"
format_stamp: "Formatted at 2026-08-06 22:50:03 on dist-test-slave-1xrh"
server_key: "ef2f89b249c2c1c574546e76c6470965"
server_key_iv: "8fea28a96f89d966f68895820e0819cb"
server_key_version: "encryptionkey@0"
I20260806 22:50:03.993925 495 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056586788339-29547-0/minicluster-data/ts-1/wal/instance:
uuid: "72dea20d69284cd9a70d4a2e0ebfd0ce"
format_stamp: "Formatted at 2026-08-06 22:50:03 on dist-test-slave-1xrh"
server_key: "ef2f89b249c2c1c574546e76c6470965"
server_key_iv: "8fea28a96f89d966f68895820e0819cb"
server_key_version: "encryptionkey@0"
I20260806 22:50:03.995208 495 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 22:50:03.995903 511 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:03.996094 495 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:50:03.996199 495 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056586788339-29547-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056586788339-29547-0/minicluster-data/ts-1/wal
uuid: "72dea20d69284cd9a70d4a2e0ebfd0ce"
format_stamp: "Formatted at 2026-08-06 22:50:03 on dist-test-slave-1xrh"
server_key: "ef2f89b249c2c1c574546e76c6470965"
server_key_iv: "8fea28a96f89d966f68895820e0819cb"
server_key_version: "encryptionkey@0"
I20260806 22:50:03.996276 495 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056586788339-29547-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056586788339-29547-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056586788339-29547-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:50:04.006546 495 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:50:04.008322 495 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:50:04.008481 495 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:50:04.008703 495 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:50:04.008998 495 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:50:04.009052 495 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:04.009092 495 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:50:04.009120 495 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:04.014719 495 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.194:37595
I20260806 22:50:04.014794 636 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.194:37595 every 8 connection(s)
I20260806 22:50:04.015100 495 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056586788339-29547-0/minicluster-data/ts-1/data/info.pb
Aug 06 22:50:04 dist-test-slave-1xrh krb5kdc[32699](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056603, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.194@KRBTEST.COM for kudu/127.28.218.254@KRBTEST.COM
I20260806 22:50:04.022583 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 495
I20260806 22:50:04.022708 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056586788339-29547-0/minicluster-data/ts-1/wal/instance
I20260806 22:50:04.022902 29547 external_mini_cluster.cc:1428] Setting key c505a39863e8ebef5e7e445cec6d234f
I20260806 22:50:04.023962 335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:04.015814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.194:33261 (local address 127.28.218.254:35269)
0806 22:50:04.015859 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:04.015861 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:04.016054 (+ 193us) server_negotiation.cc:407] Connection header received
0806 22:50:04.016507 (+ 453us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:04.016508 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:04.016517 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:04.016570 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:04.017423 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:04.017913 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:04.018883 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:04.019065 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:04.019115 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:04.021228 (+ 2113us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:50:04.021229 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:50:04.021230 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:50:04.021237 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:50:04.022506 (+ 1269us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:04.022975 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:04.022976 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:04.022977 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:04.023040 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:04.023316 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:04.023317 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:04.023317 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:04.023434 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:50:04.023515 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:04.023781 (+ 266us) server_negotiation.cc:299] Negotiation successful
0806 22:50:04.023871 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 22:50:04.024160 639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:04.015760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:35269 (local address 127.28.218.194:33261)
0806 22:50:04.015924 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:04.015963 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:04.016421 (+ 458us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:04.016622 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:04.016625 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:04.017165 (+ 540us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:04.017339 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:04.017344 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:04.018003 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:04.018004 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:04.018802 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:04.018804 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:04.019179 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:04.019182 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:04.019285 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:04.019799 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:50:04.019807 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:04.021120 (+ 1313us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:50:04.022588 (+ 1468us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:04.022595 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:04.022597 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:04.022889 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:04.023112 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:04.023113 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:04.023113 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:04.023244 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:04.023517 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:50:04.023523 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:50:04.023654 (+ 131us) client_negotiation.cc:769] Sending connection context
0806 22:50:04.023760 (+ 106us) client_negotiation.cc:240] Negotiation successful
0806 22:50:04.023921 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":90,"threads_started":1}
I20260806 22:50:04.024621 637 heartbeater.cc:347] Connected to a master server at 127.28.218.254:35269
I20260806 22:50:04.024720 637 heartbeater.cc:464] Registering TS with master...
I20260806 22:50:04.024899 637 heartbeater.cc:511] Master 127.28.218.254:35269 requested a full tablet report, sending...
I20260806 22:50:04.025288 32748 ts_manager.cc:194] Registered new tserver with Master: 72dea20d69284cd9a70d4a2e0ebfd0ce (127.28.218.194:37595)
I20260806 22:50:04.025696 32748 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.218.194@KRBTEST.COM'} at 127.28.218.194:33261
WARNING: no policy specified for kudu/127.28.218.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.218.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.218.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.195.keytab.
Entry for principal kudu/127.28.218.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.195.keytab.
I20260806 22:50:04.047820 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056586788339-29547-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056586788339-29547-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056586788339-29547-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056586788339-29547-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.218.195:0
--local_ip_for_outbound_sockets=127.28.218.195
--webserver_interface=127.28.218.195
--webserver_port=0
--tserver_master_addrs=127.28.218.254:35269
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.195.keytab
--principal=kudu/127.28.218.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.218.212:43069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:50:04.126698 642 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:50:04.126914 642 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:50:04.126951 642 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:50:04.128086 642 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:50:04.128151 642 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:50:04.128196 642 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.218.195
I20260806 22:50:04.129621 642 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:43069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056586788339-29547-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056586788339-29547-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.195.keytab
--principal=kudu/127.28.218.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.218.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056586788339-29547-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.218.195
--webserver_port=0
--tserver_master_addrs=127.28.218.254:35269
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.642
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.218.195
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056586788339-29547-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:50:04.129873 642 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:50:04.130100 642 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:50:04.134898 651 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:50:04.135004 642 server_base.cc:1034] running on GCE node
W20260806 22:50:04.134974 648 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:04.134918 649 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:50:04.135341 642 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:50:04.135581 642 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:50:04.136745 642 hybrid_clock.cc:648] HybridClock initialized: now 1786056604136745 us; error 39 us; skew 500 ppm
Aug 06 22:50:04 dist-test-slave-1xrh krb5kdc[32699](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056604, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:50:04.139216 642 init.cc:377] Logged in from keytab as kudu/127.28.218.195@KRBTEST.COM (short username kudu)
I20260806 22:50:04.139619 642 webserver.cc:460] Webserver started at http://127.28.218.195:37035/ using document root <none> and password file <none>
I20260806 22:50:04.139849 642 fs_manager.cc:359] Metadata directory not provided
I20260806 22:50:04.139917 642 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:50:04.140072 642 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:50:04.141043 642 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056586788339-29547-0/minicluster-data/ts-2/data/instance:
uuid: "9f15af3e009a43be84769557fc1e89d2"
format_stamp: "Formatted at 2026-08-06 22:50:04 on dist-test-slave-1xrh"
server_key: "2061bb67f16968a5805399e9cbceccd1"
server_key_iv: "bc7f377db496a4b86dc54c6ecb044863"
server_key_version: "encryptionkey@0"
I20260806 22:50:04.141386 642 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056586788339-29547-0/minicluster-data/ts-2/wal/instance:
uuid: "9f15af3e009a43be84769557fc1e89d2"
format_stamp: "Formatted at 2026-08-06 22:50:04 on dist-test-slave-1xrh"
server_key: "2061bb67f16968a5805399e9cbceccd1"
server_key_iv: "bc7f377db496a4b86dc54c6ecb044863"
server_key_version: "encryptionkey@0"
I20260806 22:50:04.142715 642 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:50:04.143473 658 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:04.143643 642 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:04.143739 642 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056586788339-29547-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056586788339-29547-0/minicluster-data/ts-2/wal
uuid: "9f15af3e009a43be84769557fc1e89d2"
format_stamp: "Formatted at 2026-08-06 22:50:04 on dist-test-slave-1xrh"
server_key: "2061bb67f16968a5805399e9cbceccd1"
server_key_iv: "bc7f377db496a4b86dc54c6ecb044863"
server_key_version: "encryptionkey@0"
I20260806 22:50:04.143843 642 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056586788339-29547-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056586788339-29547-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056586788339-29547-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:50:04.156610 642 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:50:04.158237 642 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:50:04.158399 642 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:50:04.158607 642 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:50:04.158888 642 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:50:04.158943 642 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:04.158975 642 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:50:04.159003 642 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:04.164743 642 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.195:38683
I20260806 22:50:04.164814 782 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.195:38683 every 8 connection(s)
I20260806 22:50:04.165098 642 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056586788339-29547-0/minicluster-data/ts-2/data/info.pb
Aug 06 22:50:04 dist-test-slave-1xrh krb5kdc[32699](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056604, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.195@KRBTEST.COM for kudu/127.28.218.254@KRBTEST.COM
I20260806 22:50:04.172240 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 642
I20260806 22:50:04.172325 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056586788339-29547-0/minicluster-data/ts-2/wal/instance
I20260806 22:50:04.172500 29547 external_mini_cluster.cc:1428] Setting key 0a4b914ddb43428faa79b3c3e1e4e6fb
I20260806 22:50:04.173738 335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:04.165796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.195:33563 (local address 127.28.218.254:35269)
0806 22:50:04.165841 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:04.165843 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:04.166005 (+ 162us) server_negotiation.cc:407] Connection header received
0806 22:50:04.166446 (+ 441us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:04.166447 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:04.166458 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:04.166508 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:04.167397 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:04.167878 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:04.168785 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:04.168971 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:04.169020 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:04.170852 (+ 1832us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:50:04.170853 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:50:04.170854 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:50:04.170860 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:50:04.172338 (+ 1478us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:04.172788 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:04.172789 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:04.172790 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:04.172855 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:04.173108 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:04.173109 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:04.173109 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:04.173236 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:50:04.173305 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:04.173615 (+ 310us) server_negotiation.cc:299] Negotiation successful
0806 22:50:04.173677 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 22:50:04.173918 785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:04.165767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:35269 (local address 127.28.218.195:33563)
0806 22:50:04.165908 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:04.165938 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:04.166356 (+ 418us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:04.166555 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:04.166558 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:04.167075 (+ 517us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:04.167323 (+ 248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:04.167327 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:04.167954 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:04.167954 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:04.168712 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:04.168713 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:04.169040 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:04.169041 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:04.169131 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:04.169516 (+ 385us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:50:04.169524 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:04.170774 (+ 1250us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:50:04.172425 (+ 1651us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:04.172431 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:04.172433 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:04.172712 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:04.172934 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:04.172935 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:04.172936 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:04.173040 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:04.173365 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:50:04.173372 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:50:04.173514 (+ 142us) client_negotiation.cc:769] Sending connection context
0806 22:50:04.173596 (+ 82us) client_negotiation.cc:240] Negotiation successful
0806 22:50:04.173703 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":86,"threads_started":1}
I20260806 22:50:04.174389 783 heartbeater.cc:347] Connected to a master server at 127.28.218.254:35269
I20260806 22:50:04.174477 783 heartbeater.cc:464] Registering TS with master...
I20260806 22:50:04.174600 783 heartbeater.cc:511] Master 127.28.218.254:35269 requested a full tablet report, sending...
I20260806 22:50:04.174940 32748 ts_manager.cc:194] Registered new tserver with Master: 9f15af3e009a43be84769557fc1e89d2 (127.28.218.195:38683)
I20260806 22:50:04.175374 32748 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.218.195@KRBTEST.COM'} at 127.28.218.195:33563
I20260806 22:50:04.184233 29547 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 22:50:04 dist-test-slave-1xrh krb5kdc[32699](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056604, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 06 22:50:04 dist-test-slave-1xrh krb5kdc[32699](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056604, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.28.218.254@KRBTEST.COM
I20260806 22:50:04.206811 335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:04.199913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55238 (local address 127.28.218.254:35269)
0806 22:50:04.199977 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:04.199979 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:04.200101 (+ 122us) server_negotiation.cc:407] Connection header received
0806 22:50:04.200180 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:04.200181 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:04.200192 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:04.200263 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:04.200836 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:04.201311 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:04.202042 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:04.202241 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:04.202342 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:04.204763 (+ 2421us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:50:04.204764 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:50:04.204765 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:50:04.204771 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:50:04.205903 (+ 1132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:04.206208 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:04.206209 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:04.206209 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:04.206257 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:04.206421 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:04.206422 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:04.206422 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:04.206541 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:50:04.206613 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:04.206696 (+ 83us) server_negotiation.cc:299] Negotiation successful
0806 22:50:04.206751 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
W20260806 22:50:04.207039 32748 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:55238
I20260806 22:50:04.207505 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 350
I20260806 22:50:04.212935 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 495
I20260806 22:50:04.219096 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 642
I20260806 22:50:04.225672 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 32713
2026-08-06T22:50:04Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (1830 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056586788339-29547-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:50:04 dist-test-slave-1xrh krb5kdc[796](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:50:04 dist-test-slave-1xrh krb5kdc[796](info): set up 2 sockets
Aug 06 22:50:04 dist-test-slave-1xrh krb5kdc[796](info): commencing operation
krb5kdc: starting...
W20260806 22:50:05.290639 29547 mini_kdc.cc:121] Time spent starting KDC: real 1.045s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056586788339-29547-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056586788339-29547-0/krb5kdc/test-user.keytab.
Aug 06 22:50:05 dist-test-slave-1xrh krb5kdc[796](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056605, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:50:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:50:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.218.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.218.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.218.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab.
Entry for principal kudu/127.28.218.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab.
I20260806 22:50:05.409197 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056586788339-29547-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056586788339-29547-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056586788339-29547-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.218.254:43687
--webserver_interface=127.28.218.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab
--principal=kudu/127.28.218.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.218.212:37179
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.218.254:43687
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:50:05.489650 810 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:50:05.489843 810 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:50:05.489866 810 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:50:05.490998 810 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:50:05.491037 810 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:50:05.491051 810 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:50:05.491101 810 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:50:05.491145 810 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:50:05.492643 810 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:37179
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056586788339-29547-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056586788339-29547-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.218.254:43687
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab
--principal=kudu/127.28.218.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.218.254:43687
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.218.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.810
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056586788339-29547-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:50:05.492887 810 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:50:05.493145 810 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:50:05.497867 818 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:05.497913 815 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:50:05.497931 810 server_base.cc:1034] running on GCE node
W20260806 22:50:05.497901 816 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:50:05.498340 810 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:50:05.498574 810 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:50:05.499706 810 hybrid_clock.cc:648] HybridClock initialized: now 1786056605499683 us; error 39 us; skew 500 ppm
Aug 06 22:50:05 dist-test-slave-1xrh krb5kdc[796](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056605, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:50:05.506855 810 init.cc:377] Logged in from keytab as kudu/127.28.218.254@KRBTEST.COM (short username kudu)
I20260806 22:50:05.507334 810 webserver.cc:460] Webserver started at http://127.28.218.254:41993/ using document root <none> and password file <none>
I20260806 22:50:05.507525 810 fs_manager.cc:359] Metadata directory not provided
I20260806 22:50:05.507565 810 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:50:05.507652 810 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:50:05.508589 810 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056586788339-29547-0/minicluster-data/master-0/data/instance:
uuid: "927ff48be830469182805f195cad09e0"
format_stamp: "Formatted at 2026-08-06 22:50:05 on dist-test-slave-1xrh"
server_key: "1e7e466f9a5bb597873fccbbe3e3157d"
server_key_iv: "f3e4fdfb7a119baa74ab63e58feac7d2"
server_key_version: "encryptionkey@0"
I20260806 22:50:05.508868 810 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056586788339-29547-0/minicluster-data/master-0/wal/instance:
uuid: "927ff48be830469182805f195cad09e0"
format_stamp: "Formatted at 2026-08-06 22:50:05 on dist-test-slave-1xrh"
server_key: "1e7e466f9a5bb597873fccbbe3e3157d"
server_key_iv: "f3e4fdfb7a119baa74ab63e58feac7d2"
server_key_version: "encryptionkey@0"
I20260806 22:50:05.510046 810 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:50:05.510715 826 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:05.510879 810 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:50:05.510993 810 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056586788339-29547-0/minicluster-data/master-0/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056586788339-29547-0/minicluster-data/master-0/wal
uuid: "927ff48be830469182805f195cad09e0"
format_stamp: "Formatted at 2026-08-06 22:50:05 on dist-test-slave-1xrh"
server_key: "1e7e466f9a5bb597873fccbbe3e3157d"
server_key_iv: "f3e4fdfb7a119baa74ab63e58feac7d2"
server_key_version: "encryptionkey@0"
I20260806 22:50:05.511070 810 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056586788339-29547-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056586788339-29547-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056586788339-29547-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:50:05.528990 810 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:50:05.530644 810 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:50:05.530800 810 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:50:05.534852 810 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.254:43687
I20260806 22:50:05.534885 883 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.254:43687 every 8 connection(s)
I20260806 22:50:05.535389 810 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
I20260806 22:50:05.536038 884 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:50:05.538101 884 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 927ff48be830469182805f195cad09e0: Bootstrap starting.
I20260806 22:50:05.538632 884 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 927ff48be830469182805f195cad09e0: Neither blocks nor log segments found. Creating new log.
I20260806 22:50:05.538862 884 log.cc:824] T 00000000000000000000000000000000 P 927ff48be830469182805f195cad09e0: Log is configured to *not* fsync() on all Append() calls
I20260806 22:50:05.539546 884 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 927ff48be830469182805f195cad09e0: No bootstrap required, opened a new log
I20260806 22:50:05.540966 884 raft_consensus.cc:348] T 00000000000000000000000000000000 P 927ff48be830469182805f195cad09e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "927ff48be830469182805f195cad09e0" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 43687 } }
I20260806 22:50:05.541090 884 raft_consensus.cc:374] T 00000000000000000000000000000000 P 927ff48be830469182805f195cad09e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:50:05.541110 884 raft_consensus.cc:729] T 00000000000000000000000000000000 P 927ff48be830469182805f195cad09e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 927ff48be830469182805f195cad09e0, State: Initialized, Role: FOLLOWER
I20260806 22:50:05.541234 884 consensus_queue.cc:260] T 00000000000000000000000000000000 P 927ff48be830469182805f195cad09e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "927ff48be830469182805f195cad09e0" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 43687 } }
I20260806 22:50:05.541313 884 raft_consensus.cc:388] T 00000000000000000000000000000000 P 927ff48be830469182805f195cad09e0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:50:05.541337 884 raft_consensus.cc:482] T 00000000000000000000000000000000 P 927ff48be830469182805f195cad09e0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:50:05.541363 884 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 927ff48be830469182805f195cad09e0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:50:05.542135 884 raft_consensus.cc:504] T 00000000000000000000000000000000 P 927ff48be830469182805f195cad09e0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "927ff48be830469182805f195cad09e0" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 43687 } }
I20260806 22:50:05.542235 884 leader_election.cc:302] T 00000000000000000000000000000000 P 927ff48be830469182805f195cad09e0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 927ff48be830469182805f195cad09e0; no voters:
I20260806 22:50:05.542351 884 leader_election.cc:288] T 00000000000000000000000000000000 P 927ff48be830469182805f195cad09e0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:50:05.542423 888 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 927ff48be830469182805f195cad09e0 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:50:05.542584 884 sys_catalog.cc:564] T 00000000000000000000000000000000 P 927ff48be830469182805f195cad09e0 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:50:05.542604 888 raft_consensus.cc:686] T 00000000000000000000000000000000 P 927ff48be830469182805f195cad09e0 [term 1 LEADER]: Becoming Leader. State: Replica: 927ff48be830469182805f195cad09e0, State: Running, Role: LEADER
I20260806 22:50:05.542843 888 consensus_queue.cc:237] T 00000000000000000000000000000000 P 927ff48be830469182805f195cad09e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "927ff48be830469182805f195cad09e0" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 43687 } }
I20260806 22:50:05.543229 889 sys_catalog.cc:455] T 00000000000000000000000000000000 P 927ff48be830469182805f195cad09e0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "927ff48be830469182805f195cad09e0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "927ff48be830469182805f195cad09e0" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 43687 } } }
I20260806 22:50:05.543310 889 sys_catalog.cc:458] T 00000000000000000000000000000000 P 927ff48be830469182805f195cad09e0 [sys.catalog]: This master's current role is: LEADER
I20260806 22:50:05.543484 890 sys_catalog.cc:455] T 00000000000000000000000000000000 P 927ff48be830469182805f195cad09e0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 927ff48be830469182805f195cad09e0. Latest consensus state: current_term: 1 leader_uuid: "927ff48be830469182805f195cad09e0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "927ff48be830469182805f195cad09e0" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 43687 } } }
I20260806 22:50:05.543522 898 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:50:05.543541 890 sys_catalog.cc:458] T 00000000000000000000000000000000 P 927ff48be830469182805f195cad09e0 [sys.catalog]: This master's current role is: LEADER
I20260806 22:50:05.543836 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 810
I20260806 22:50:05.543933 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056586788339-29547-0/minicluster-data/master-0/wal/instance
I20260806 22:50:05.544121 29547 external_mini_cluster.cc:1428] Setting key 34546c45b0719fbdad15e691c9c93f57
I20260806 22:50:05.544595 898 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:50:05.546240 898 catalog_manager.cc:1347] Generated new cluster ID: 64bf9190a59c480ca545906b7236bd4d
I20260806 22:50:05.546298 898 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 22:50:05 dist-test-slave-1xrh krb5kdc[796](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056605, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.218.254@KRBTEST.COM
I20260806 22:50:05.555657 910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:05.544562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54896 (local address 127.28.218.254:43687)
0806 22:50:05.544859 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:05.544868 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:05.544899 (+ 31us) server_negotiation.cc:407] Connection header received
0806 22:50:05.545233 (+ 334us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:05.545242 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:05.545305 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:05.545482 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:05.546203 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:05.547622 (+ 1419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:05.548698 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:05.549298 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:05.549398 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:05.552115 (+ 2717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:50:05.552119 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:50:05.552119 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:50:05.552131 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:50:05.554016 (+ 1885us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:05.554419 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:05.554422 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:05.554423 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:05.554531 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:05.554730 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:05.554731 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:05.554731 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:05.555044 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:50:05.555234 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:05.555306 (+ 72us) server_negotiation.cc:299] Negotiation successful
0806 22:50:05.555451 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":259,"thread_start_us":71,"threads_started":1}
I20260806 22:50:05.565631 898 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:50:05.566288 898 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:50:05.578508 898 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 927ff48be830469182805f195cad09e0: Generated new TSK 0
I20260806 22:50:05.578696 898 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.28.218.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.218.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.218.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.193.keytab.
Entry for principal kudu/127.28.218.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.193.keytab.
I20260806 22:50:05.610297 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056586788339-29547-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056586788339-29547-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056586788339-29547-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056586788339-29547-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.218.193:0
--local_ip_for_outbound_sockets=127.28.218.193
--webserver_interface=127.28.218.193
--webserver_port=0
--tserver_master_addrs=127.28.218.254:43687
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.193.keytab
--principal=kudu/127.28.218.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.218.212:37179
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:50:05.690431 915 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:50:05.690673 915 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:50:05.690737 915 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:50:05.691944 915 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:50:05.692009 915 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:50:05.692049 915 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.218.193
I20260806 22:50:05.693527 915 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:37179
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056586788339-29547-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056586788339-29547-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.193.keytab
--principal=kudu/127.28.218.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.218.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056586788339-29547-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.218.193
--webserver_port=0
--tserver_master_addrs=127.28.218.254:43687
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.915
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.218.193
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056586788339-29547-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:50:05.693785 915 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:50:05.694002 915 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:50:05.698755 925 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:05.698777 921 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:50:05.698820 915 server_base.cc:1034] running on GCE node
W20260806 22:50:05.698787 922 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:50:05.699218 915 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:50:05.699429 915 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:50:05.700567 915 hybrid_clock.cc:648] HybridClock initialized: now 1786056605700548 us; error 33 us; skew 500 ppm
Aug 06 22:50:05 dist-test-slave-1xrh krb5kdc[796](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056605, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:50:05.703037 915 init.cc:377] Logged in from keytab as kudu/127.28.218.193@KRBTEST.COM (short username kudu)
I20260806 22:50:05.703473 915 webserver.cc:460] Webserver started at http://127.28.218.193:44727/ using document root <none> and password file <none>
I20260806 22:50:05.703696 915 fs_manager.cc:359] Metadata directory not provided
I20260806 22:50:05.703738 915 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:50:05.703886 915 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:50:05.704867 915 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056586788339-29547-0/minicluster-data/ts-0/data/instance:
uuid: "94c6a158e7d54995b85bd2f1ac7b9a03"
format_stamp: "Formatted at 2026-08-06 22:50:05 on dist-test-slave-1xrh"
server_key: "5f3e4177a0b55e53473439e344c94c02"
server_key_iv: "1a7c16ab754a706f87bc17d003d7c20f"
server_key_version: "encryptionkey@0"
I20260806 22:50:05.705217 915 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056586788339-29547-0/minicluster-data/ts-0/wal/instance:
uuid: "94c6a158e7d54995b85bd2f1ac7b9a03"
format_stamp: "Formatted at 2026-08-06 22:50:05 on dist-test-slave-1xrh"
server_key: "5f3e4177a0b55e53473439e344c94c02"
server_key_iv: "1a7c16ab754a706f87bc17d003d7c20f"
server_key_version: "encryptionkey@0"
I20260806 22:50:05.706480 915 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:50:05.707192 932 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:05.707362 915 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:50:05.707468 915 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056586788339-29547-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056586788339-29547-0/minicluster-data/ts-0/wal
uuid: "94c6a158e7d54995b85bd2f1ac7b9a03"
format_stamp: "Formatted at 2026-08-06 22:50:05 on dist-test-slave-1xrh"
server_key: "5f3e4177a0b55e53473439e344c94c02"
server_key_iv: "1a7c16ab754a706f87bc17d003d7c20f"
server_key_version: "encryptionkey@0"
I20260806 22:50:05.707546 915 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056586788339-29547-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056586788339-29547-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056586788339-29547-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:50:05.725327 915 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:50:05.727097 915 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:50:05.727313 915 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:50:05.727555 915 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:50:05.727877 915 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:50:05.727952 915 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:05.727994 915 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:50:05.728027 915 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:05.733753 915 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.193:43451
I20260806 22:50:05.733803 1055 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.193:43451 every 8 connection(s)
I20260806 22:50:05.734103 915 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056586788339-29547-0/minicluster-data/ts-0/data/info.pb
I20260806 22:50:05.735317 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 915
I20260806 22:50:05.735430 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056586788339-29547-0/minicluster-data/ts-0/wal/instance
I20260806 22:50:05.735633 29547 external_mini_cluster.cc:1428] Setting key 75146b5d8a9f74796d1e13c96ee36628
Aug 06 22:50:05 dist-test-slave-1xrh krb5kdc[796](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056605, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.193@KRBTEST.COM for kudu/127.28.218.254@KRBTEST.COM
I20260806 22:50:05.744511 910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:05.734803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.193:33029 (local address 127.28.218.254:43687)
0806 22:50:05.734881 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:05.734883 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:05.735032 (+ 149us) server_negotiation.cc:407] Connection header received
0806 22:50:05.736376 (+ 1344us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:05.736377 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:05.736391 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:05.736454 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:05.737279 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:05.737763 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:05.738824 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:05.739018 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:05.739071 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:05.741492 (+ 2421us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:50:05.741494 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:50:05.741494 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:50:05.741502 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:50:05.742977 (+ 1475us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:05.743444 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:05.743446 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:05.743446 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:05.743506 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:05.743798 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:05.743810 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:05.743810 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:05.743950 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:50:05.744025 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:05.744305 (+ 280us) server_negotiation.cc:299] Negotiation successful
0806 22:50:05.744396 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 22:50:05.744657 1058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:05.734755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:43687 (local address 127.28.218.193:33029)
0806 22:50:05.734917 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:05.734951 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:05.736275 (+ 1324us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:05.736537 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:05.736541 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:05.736922 (+ 381us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:05.737194 (+ 272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:05.737200 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:05.737878 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:05.737879 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:05.738756 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:05.738758 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:05.739107 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:05.739108 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:05.739280 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:05.739767 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:50:05.739776 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:05.741343 (+ 1567us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:50:05.743061 (+ 1718us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:05.743069 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:05.743071 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:05.743364 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:05.743576 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:05.743577 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:05.743578 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:05.743724 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:05.744028 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:50:05.744036 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:50:05.744167 (+ 131us) client_negotiation.cc:769] Sending connection context
0806 22:50:05.744267 (+ 100us) client_negotiation.cc:240] Negotiation successful
0806 22:50:05.744433 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":92,"threads_started":1}
I20260806 22:50:05.745095 1056 heartbeater.cc:347] Connected to a master server at 127.28.218.254:43687
I20260806 22:50:05.745180 1056 heartbeater.cc:464] Registering TS with master...
I20260806 22:50:05.745354 1056 heartbeater.cc:511] Master 127.28.218.254:43687 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.28.218.194@KRBTEST.COM; defaulting to no policy
I20260806 22:50:05.745870 844 ts_manager.cc:194] Registered new tserver with Master: 94c6a158e7d54995b85bd2f1ac7b9a03 (127.28.218.193:43451)
I20260806 22:50:05.746476 844 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.218.193@KRBTEST.COM'} at 127.28.218.193:33029
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.218.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.218.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.194.keytab.
Entry for principal kudu/127.28.218.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.194.keytab.
I20260806 22:50:05.763613 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056586788339-29547-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056586788339-29547-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056586788339-29547-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056586788339-29547-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.218.194:0
--local_ip_for_outbound_sockets=127.28.218.194
--webserver_interface=127.28.218.194
--webserver_port=0
--tserver_master_addrs=127.28.218.254:43687
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.194.keytab
--principal=kudu/127.28.218.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.218.212:37179
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:50:05.843989 1061 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:50:05.844184 1061 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:50:05.844206 1061 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:50:05.845335 1061 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:50:05.845376 1061 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:50:05.845438 1061 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.218.194
I20260806 22:50:05.846926 1061 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:37179
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056586788339-29547-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056586788339-29547-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.194.keytab
--principal=kudu/127.28.218.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.218.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056586788339-29547-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.218.194
--webserver_port=0
--tserver_master_addrs=127.28.218.254:43687
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1061
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.218.194
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056586788339-29547-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:50:05.847182 1061 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:50:05.847415 1061 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:50:05.852283 1070 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:50:05.852312 1061 server_base.cc:1034] running on GCE node
W20260806 22:50:05.852321 1067 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:05.852284 1068 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:50:05.852795 1061 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:50:05.853042 1061 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:50:05.854185 1061 hybrid_clock.cc:648] HybridClock initialized: now 1786056605854174 us; error 30 us; skew 500 ppm
Aug 06 22:50:05 dist-test-slave-1xrh krb5kdc[796](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056605, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:50:05.856576 1061 init.cc:377] Logged in from keytab as kudu/127.28.218.194@KRBTEST.COM (short username kudu)
I20260806 22:50:05.856979 1061 webserver.cc:460] Webserver started at http://127.28.218.194:37349/ using document root <none> and password file <none>
I20260806 22:50:05.857191 1061 fs_manager.cc:359] Metadata directory not provided
I20260806 22:50:05.857256 1061 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:50:05.857405 1061 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:50:05.858376 1061 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056586788339-29547-0/minicluster-data/ts-1/data/instance:
uuid: "420da685f2604fadb404edaa1972196f"
format_stamp: "Formatted at 2026-08-06 22:50:05 on dist-test-slave-1xrh"
server_key: "d9deacee5be823068e1e8af44fad7581"
server_key_iv: "186c0610df9f8a2cfe94a3f6c7c6f17c"
server_key_version: "encryptionkey@0"
I20260806 22:50:05.858726 1061 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056586788339-29547-0/minicluster-data/ts-1/wal/instance:
uuid: "420da685f2604fadb404edaa1972196f"
format_stamp: "Formatted at 2026-08-06 22:50:05 on dist-test-slave-1xrh"
server_key: "d9deacee5be823068e1e8af44fad7581"
server_key_iv: "186c0610df9f8a2cfe94a3f6c7c6f17c"
server_key_version: "encryptionkey@0"
I20260806 22:50:05.860111 1061 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:50:05.860752 1080 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:05.860905 1061 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:50:05.861014 1061 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056586788339-29547-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056586788339-29547-0/minicluster-data/ts-1/wal
uuid: "420da685f2604fadb404edaa1972196f"
format_stamp: "Formatted at 2026-08-06 22:50:05 on dist-test-slave-1xrh"
server_key: "d9deacee5be823068e1e8af44fad7581"
server_key_iv: "186c0610df9f8a2cfe94a3f6c7c6f17c"
server_key_version: "encryptionkey@0"
I20260806 22:50:05.861083 1061 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056586788339-29547-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056586788339-29547-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056586788339-29547-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:50:05.880425 1061 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:50:05.882138 1061 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:50:05.882287 1061 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:50:05.882496 1061 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:50:05.882779 1061 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:50:05.882831 1061 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:05.882869 1061 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:50:05.882896 1061 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:05.888418 1061 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.194:41957
I20260806 22:50:05.888459 1203 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.194:41957 every 8 connection(s)
I20260806 22:50:05.888759 1061 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056586788339-29547-0/minicluster-data/ts-1/data/info.pb
Aug 06 22:50:05 dist-test-slave-1xrh krb5kdc[796](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056605, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.194@KRBTEST.COM for kudu/127.28.218.254@KRBTEST.COM
I20260806 22:50:05.897467 910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:05.889434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.194:33675 (local address 127.28.218.254:43687)
0806 22:50:05.889519 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:05.889521 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:05.889692 (+ 171us) server_negotiation.cc:407] Connection header received
0806 22:50:05.890657 (+ 965us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:05.890658 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:05.890671 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:05.890716 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:05.891372 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:05.891859 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:05.892806 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:05.893017 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:05.893101 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:05.895082 (+ 1981us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:50:05.895083 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:50:05.895084 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:50:05.895090 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:50:05.896360 (+ 1270us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:05.896709 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:05.896710 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:05.896710 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:05.896758 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:05.896977 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:05.896978 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:05.896978 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:05.897075 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:50:05.897123 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:05.897342 (+ 219us) server_negotiation.cc:299] Negotiation successful
0806 22:50:05.897404 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 22:50:05.897605 1206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:05.889379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:43687 (local address 127.28.218.194:33675)
0806 22:50:05.889569 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:05.889603 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:05.890536 (+ 933us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:05.890790 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:05.890793 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:05.891037 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:05.891275 (+ 238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:05.891279 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:05.891961 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:05.891962 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:05.892717 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:05.892720 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:05.893108 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:05.893110 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:05.893238 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:05.893614 (+ 376us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:50:05.893622 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:05.895001 (+ 1379us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:50:05.896430 (+ 1429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:05.896435 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:05.896437 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:05.896647 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:05.896815 (+ 168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:05.896816 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:05.896816 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:05.896915 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:05.897129 (+ 214us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:50:05.897133 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:50:05.897238 (+ 105us) client_negotiation.cc:769] Sending connection context
0806 22:50:05.897315 (+ 77us) client_negotiation.cc:240] Negotiation successful
0806 22:50:05.897423 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":109,"threads_started":1}
I20260806 22:50:05.898073 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 1061
I20260806 22:50:05.898111 1204 heartbeater.cc:347] Connected to a master server at 127.28.218.254:43687
I20260806 22:50:05.898197 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056586788339-29547-0/minicluster-data/ts-1/wal/instance
I20260806 22:50:05.898221 1204 heartbeater.cc:464] Registering TS with master...
I20260806 22:50:05.898365 29547 external_mini_cluster.cc:1428] Setting key f3f486c471c2092ca434a0de65875fab
I20260806 22:50:05.898384 1204 heartbeater.cc:511] Master 127.28.218.254:43687 requested a full tablet report, sending...
I20260806 22:50:05.898748 843 ts_manager.cc:194] Registered new tserver with Master: 420da685f2604fadb404edaa1972196f (127.28.218.194:41957)
I20260806 22:50:05.899235 843 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.218.194@KRBTEST.COM'} at 127.28.218.194:33675
WARNING: no policy specified for kudu/127.28.218.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.218.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.218.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.195.keytab.
Entry for principal kudu/127.28.218.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.195.keytab.
I20260806 22:50:05.925810 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056586788339-29547-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056586788339-29547-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056586788339-29547-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056586788339-29547-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.218.195:0
--local_ip_for_outbound_sockets=127.28.218.195
--webserver_interface=127.28.218.195
--webserver_port=0
--tserver_master_addrs=127.28.218.254:43687
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.195.keytab
--principal=kudu/127.28.218.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.218.212:37179
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:50:06.005004 1211 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:50:06.005236 1211 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:50:06.005271 1211 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:50:06.006384 1211 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:50:06.006449 1211 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:50:06.006505 1211 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.218.195
I20260806 22:50:06.008073 1211 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:37179
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056586788339-29547-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056586788339-29547-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.195.keytab
--principal=kudu/127.28.218.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.218.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056586788339-29547-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.218.195
--webserver_port=0
--tserver_master_addrs=127.28.218.254:43687
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1211
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.218.195
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056586788339-29547-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:50:06.008324 1211 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:50:06.008553 1211 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:50:06.013568 1217 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:06.013621 1216 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:06.013581 1219 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:50:06.013844 1211 server_base.cc:1034] running on GCE node
I20260806 22:50:06.014066 1211 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:50:06.014343 1211 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:50:06.015486 1211 hybrid_clock.cc:648] HybridClock initialized: now 1786056606015476 us; error 50 us; skew 500 ppm
Aug 06 22:50:06 dist-test-slave-1xrh krb5kdc[796](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056606, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:50:06.017961 1211 init.cc:377] Logged in from keytab as kudu/127.28.218.195@KRBTEST.COM (short username kudu)
I20260806 22:50:06.018375 1211 webserver.cc:460] Webserver started at http://127.28.218.195:40451/ using document root <none> and password file <none>
I20260806 22:50:06.018610 1211 fs_manager.cc:359] Metadata directory not provided
I20260806 22:50:06.018678 1211 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:50:06.018827 1211 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:50:06.019906 1211 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056586788339-29547-0/minicluster-data/ts-2/data/instance:
uuid: "13fe15d297b2404fb817e561e2d6b946"
format_stamp: "Formatted at 2026-08-06 22:50:06 on dist-test-slave-1xrh"
server_key: "6eadbe21827557825f844850efb27a27"
server_key_iv: "b24d6ff20d9520e59da84501f29703e2"
server_key_version: "encryptionkey@0"
I20260806 22:50:06.020334 1211 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056586788339-29547-0/minicluster-data/ts-2/wal/instance:
uuid: "13fe15d297b2404fb817e561e2d6b946"
format_stamp: "Formatted at 2026-08-06 22:50:06 on dist-test-slave-1xrh"
server_key: "6eadbe21827557825f844850efb27a27"
server_key_iv: "b24d6ff20d9520e59da84501f29703e2"
server_key_version: "encryptionkey@0"
I20260806 22:50:06.021651 1211 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:50:06.022351 1227 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:06.022521 1211 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:50:06.022622 1211 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056586788339-29547-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056586788339-29547-0/minicluster-data/ts-2/wal
uuid: "13fe15d297b2404fb817e561e2d6b946"
format_stamp: "Formatted at 2026-08-06 22:50:06 on dist-test-slave-1xrh"
server_key: "6eadbe21827557825f844850efb27a27"
server_key_iv: "b24d6ff20d9520e59da84501f29703e2"
server_key_version: "encryptionkey@0"
I20260806 22:50:06.022708 1211 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056586788339-29547-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056586788339-29547-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056586788339-29547-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:50:06.056035 1211 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:50:06.058398 1211 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:50:06.058513 1211 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:50:06.058672 1211 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:50:06.058969 1211 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:50:06.059023 1211 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:06.059063 1211 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:50:06.059091 1211 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:06.064890 1211 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.195:38943
I20260806 22:50:06.064939 1346 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.195:38943 every 8 connection(s)
I20260806 22:50:06.065272 1211 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056586788339-29547-0/minicluster-data/ts-2/data/info.pb
I20260806 22:50:06.070729 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 1211
I20260806 22:50:06.070822 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056586788339-29547-0/minicluster-data/ts-2/wal/instance
I20260806 22:50:06.071013 29547 external_mini_cluster.cc:1428] Setting key 4487940ba85f7da875ae627ac598500d
Aug 06 22:50:06 dist-test-slave-1xrh krb5kdc[796](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056606, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.195@KRBTEST.COM for kudu/127.28.218.254@KRBTEST.COM
I20260806 22:50:06.074397 910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:06.066013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.195:56531 (local address 127.28.218.254:43687)
0806 22:50:06.066074 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:06.066076 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:06.066257 (+ 181us) server_negotiation.cc:407] Connection header received
0806 22:50:06.067210 (+ 953us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:06.067211 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:06.067229 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:06.067279 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:06.067841 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:06.068316 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:06.069236 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:06.069411 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:06.069461 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:06.071778 (+ 2317us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:50:06.071779 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:50:06.071779 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:50:06.071785 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:50:06.073272 (+ 1487us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:06.073634 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:06.073635 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:06.073635 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:06.073697 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:06.073919 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:06.073919 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:06.073920 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:06.074020 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:50:06.074069 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:06.074273 (+ 204us) server_negotiation.cc:299] Negotiation successful
0806 22:50:06.074336 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260806 22:50:06.074548 1351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:06.066013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:43687 (local address 127.28.218.195:56531)
0806 22:50:06.066158 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:06.066191 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:06.067069 (+ 878us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:06.067329 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:06.067332 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:06.067573 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:06.067767 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:06.067771 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:06.068389 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:06.068390 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:06.069161 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:06.069163 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:06.069480 (+ 317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:06.069480 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:06.069577 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:06.070058 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:50:06.070066 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:06.071688 (+ 1622us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:50:06.073339 (+ 1651us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:06.073344 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:06.073346 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:06.073566 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:06.073751 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:06.073752 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:06.073752 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:06.073858 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:06.074074 (+ 216us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:50:06.074077 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:50:06.074177 (+ 100us) client_negotiation.cc:769] Sending connection context
0806 22:50:06.074258 (+ 81us) client_negotiation.cc:240] Negotiation successful
0806 22:50:06.074365 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":75,"threads_started":1}
I20260806 22:50:06.075011 1347 heartbeater.cc:347] Connected to a master server at 127.28.218.254:43687
I20260806 22:50:06.075111 1347 heartbeater.cc:464] Registering TS with master...
I20260806 22:50:06.075304 1347 heartbeater.cc:511] Master 127.28.218.254:43687 requested a full tablet report, sending...
I20260806 22:50:06.075713 843 ts_manager.cc:194] Registered new tserver with Master: 13fe15d297b2404fb817e561e2d6b946 (127.28.218.195:38943)
I20260806 22:50:06.076083 843 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.218.195@KRBTEST.COM'} at 127.28.218.195:56531
I20260806 22:50:06.082967 29547 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 22:50:06 dist-test-slave-1xrh krb5kdc[796](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056606, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 22:50:06 dist-test-slave-1xrh krb5kdc[796](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056606, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.218.193@KRBTEST.COM
I20260806 22:50:06.105243 1356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:06.098093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59718 (local address 127.28.218.193:43451)
0806 22:50:06.098245 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:06.098249 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:06.098259 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:50:06.098294 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:06.098296 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:06.098308 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:06.098372 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:06.098902 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:06.099706 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:06.100400 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:06.100695 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:06.100746 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:06.102723 (+ 1977us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:50:06.102731 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:50:06.102731 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:50:06.102736 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:50:06.104339 (+ 1603us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:06.104628 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:06.104631 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:06.104631 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:06.104696 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:06.104836 (+ 140us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:06.104837 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:06.104837 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:06.104949 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:50:06.105041 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:06.105109 (+ 68us) server_negotiation.cc:299] Negotiation successful
0806 22:50:06.105168 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":81,"threads_started":1}
Aug 06 22:50:06 dist-test-slave-1xrh krb5kdc[796](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056606, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.218.254@KRBTEST.COM
I20260806 22:50:06.112640 910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:06.106509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54902 (local address 127.28.218.254:43687)
0806 22:50:06.106566 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:06.106568 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:06.106576 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:50:06.106633 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:06.106634 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:06.106642 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:06.106674 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:06.107088 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:06.107618 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:06.108221 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:06.108398 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:06.108446 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:06.110564 (+ 2118us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:50:06.110564 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:50:06.110565 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:50:06.110570 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:50:06.111767 (+ 1197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:06.112036 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:06.112037 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:06.112037 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:06.112080 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:06.112211 (+ 131us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:06.112211 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:06.112211 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:06.112319 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:50:06.112398 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:06.112487 (+ 89us) server_negotiation.cc:299] Negotiation successful
0806 22:50:06.112541 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
W20260806 22:50:06.113016 843 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54902
I20260806 22:50:06.113391 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 915
I20260806 22:50:06.122432 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 1061
I20260806 22:50:06.129575 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 1211
I20260806 22:50:06.134696 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 810
2026-08-06T22:50:06Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (1909 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056586788339-29547-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:50:06 dist-test-slave-1xrh krb5kdc[1361](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:50:06 dist-test-slave-1xrh krb5kdc[1361](info): set up 2 sockets
Aug 06 22:50:06 dist-test-slave-1xrh krb5kdc[1361](info): commencing operation
krb5kdc: starting...
W20260806 22:50:07.198158 29547 mini_kdc.cc:121] Time spent starting KDC: real 1.042s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056586788339-29547-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056586788339-29547-0/krb5kdc/test-user.keytab.
Aug 06 22:50:07 dist-test-slave-1xrh krb5kdc[1361](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056607, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:50:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:50:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.218.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.218.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.218.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab.
Entry for principal kudu/127.28.218.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab.
I20260806 22:50:07.316031 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056586788339-29547-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056586788339-29547-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056586788339-29547-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.218.254:34595
--webserver_interface=127.28.218.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab
--principal=kudu/127.28.218.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.218.212:46601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.218.254:34595
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:50:07.396476 1374 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:50:07.396665 1374 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:50:07.396687 1374 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:50:07.397820 1374 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:50:07.397861 1374 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:50:07.397876 1374 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:50:07.397893 1374 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:50:07.397907 1374 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:50:07.399456 1374 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:46601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056586788339-29547-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056586788339-29547-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.218.254:34595
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab
--principal=kudu/127.28.218.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.218.254:34595
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.218.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1374
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056586788339-29547-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:50:07.399715 1374 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:50:07.399950 1374 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:50:07.405151 1382 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:07.405180 1380 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:07.405134 1379 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:50:07.405424 1374 server_base.cc:1034] running on GCE node
I20260806 22:50:07.405628 1374 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:50:07.405845 1374 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:50:07.406980 1374 hybrid_clock.cc:648] HybridClock initialized: now 1786056607406959 us; error 39 us; skew 500 ppm
Aug 06 22:50:07 dist-test-slave-1xrh krb5kdc[1361](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056607, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:50:07.409374 1374 init.cc:377] Logged in from keytab as kudu/127.28.218.254@KRBTEST.COM (short username kudu)
I20260806 22:50:07.409648 1374 fs_manager.cc:359] Metadata directory not provided
I20260806 22:50:07.409690 1374 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:50:07.409776 1374 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:50:07.410715 1374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056586788339-29547-0/minicluster-data/master-0/data/instance:
uuid: "c4a45111206a414eaae72449b43caeb0"
format_stamp: "Formatted at 2026-08-06 22:50:07 on dist-test-slave-1xrh"
server_key: "d05f954e3c39aa5ea8f0eda4eee10ce2"
server_key_iv: "9417181329fa6db82692ddda4e733927"
server_key_version: "encryptionkey@0"
I20260806 22:50:07.411010 1374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056586788339-29547-0/minicluster-data/master-0/wal/instance:
uuid: "c4a45111206a414eaae72449b43caeb0"
format_stamp: "Formatted at 2026-08-06 22:50:07 on dist-test-slave-1xrh"
server_key: "d05f954e3c39aa5ea8f0eda4eee10ce2"
server_key_iv: "9417181329fa6db82692ddda4e733927"
server_key_version: "encryptionkey@0"
I20260806 22:50:07.412227 1374 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:50:07.412844 1389 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:07.413007 1374 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:50:07.413101 1374 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056586788339-29547-0/minicluster-data/master-0/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056586788339-29547-0/minicluster-data/master-0/wal
uuid: "c4a45111206a414eaae72449b43caeb0"
format_stamp: "Formatted at 2026-08-06 22:50:07 on dist-test-slave-1xrh"
server_key: "d05f954e3c39aa5ea8f0eda4eee10ce2"
server_key_iv: "9417181329fa6db82692ddda4e733927"
server_key_version: "encryptionkey@0"
I20260806 22:50:07.413168 1374 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056586788339-29547-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056586788339-29547-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056586788339-29547-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:50:07.437778 1374 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:50:07.439541 1374 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:50:07.439688 1374 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:50:07.443356 1374 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.254:34595
I20260806 22:50:07.443382 1443 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.254:34595 every 8 connection(s)
I20260806 22:50:07.443638 1374 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
I20260806 22:50:07.444626 1444 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:50:07.446731 1444 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4a45111206a414eaae72449b43caeb0: Bootstrap starting.
I20260806 22:50:07.447304 1444 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c4a45111206a414eaae72449b43caeb0: Neither blocks nor log segments found. Creating new log.
I20260806 22:50:07.447554 1444 log.cc:824] T 00000000000000000000000000000000 P c4a45111206a414eaae72449b43caeb0: Log is configured to *not* fsync() on all Append() calls
I20260806 22:50:07.448141 1444 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4a45111206a414eaae72449b43caeb0: No bootstrap required, opened a new log
I20260806 22:50:07.449620 1444 raft_consensus.cc:348] T 00000000000000000000000000000000 P c4a45111206a414eaae72449b43caeb0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4a45111206a414eaae72449b43caeb0" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 34595 } }
I20260806 22:50:07.449745 1444 raft_consensus.cc:374] T 00000000000000000000000000000000 P c4a45111206a414eaae72449b43caeb0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:50:07.449767 1444 raft_consensus.cc:729] T 00000000000000000000000000000000 P c4a45111206a414eaae72449b43caeb0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4a45111206a414eaae72449b43caeb0, State: Initialized, Role: FOLLOWER
I20260806 22:50:07.449898 1444 consensus_queue.cc:260] T 00000000000000000000000000000000 P c4a45111206a414eaae72449b43caeb0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4a45111206a414eaae72449b43caeb0" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 34595 } }
I20260806 22:50:07.449975 1444 raft_consensus.cc:388] T 00000000000000000000000000000000 P c4a45111206a414eaae72449b43caeb0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:50:07.449997 1444 raft_consensus.cc:482] T 00000000000000000000000000000000 P c4a45111206a414eaae72449b43caeb0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:50:07.450024 1444 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c4a45111206a414eaae72449b43caeb0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:50:07.450762 1444 raft_consensus.cc:504] T 00000000000000000000000000000000 P c4a45111206a414eaae72449b43caeb0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4a45111206a414eaae72449b43caeb0" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 34595 } }
I20260806 22:50:07.450863 1444 leader_election.cc:302] T 00000000000000000000000000000000 P c4a45111206a414eaae72449b43caeb0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c4a45111206a414eaae72449b43caeb0; no voters:
I20260806 22:50:07.451038 1444 leader_election.cc:288] T 00000000000000000000000000000000 P c4a45111206a414eaae72449b43caeb0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:50:07.451045 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 1374
I20260806 22:50:07.451089 1448 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c4a45111206a414eaae72449b43caeb0 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:50:07.451175 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056586788339-29547-0/minicluster-data/master-0/wal/instance
I20260806 22:50:07.451267 1448 raft_consensus.cc:686] T 00000000000000000000000000000000 P c4a45111206a414eaae72449b43caeb0 [term 1 LEADER]: Becoming Leader. State: Replica: c4a45111206a414eaae72449b43caeb0, State: Running, Role: LEADER
I20260806 22:50:07.451323 1444 sys_catalog.cc:564] T 00000000000000000000000000000000 P c4a45111206a414eaae72449b43caeb0 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:50:07.451366 29547 external_mini_cluster.cc:1428] Setting key fa75bf641613807482dac78ec4cb26c8
I20260806 22:50:07.451359 1448 consensus_queue.cc:237] T 00000000000000000000000000000000 P c4a45111206a414eaae72449b43caeb0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4a45111206a414eaae72449b43caeb0" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 34595 } }
I20260806 22:50:07.451678 1448 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4a45111206a414eaae72449b43caeb0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c4a45111206a414eaae72449b43caeb0. Latest consensus state: current_term: 1 leader_uuid: "c4a45111206a414eaae72449b43caeb0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4a45111206a414eaae72449b43caeb0" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 34595 } } }
I20260806 22:50:07.451769 1448 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4a45111206a414eaae72449b43caeb0 [sys.catalog]: This master's current role is: LEADER
I20260806 22:50:07.451689 1450 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4a45111206a414eaae72449b43caeb0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c4a45111206a414eaae72449b43caeb0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4a45111206a414eaae72449b43caeb0" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 34595 } } }
I20260806 22:50:07.451906 1450 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4a45111206a414eaae72449b43caeb0 [sys.catalog]: This master's current role is: LEADER
W20260806 22:50:07.453492 1466 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c4a45111206a414eaae72449b43caeb0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 22:50:07.453552 1466 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 22:50:07.453606 1455 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:50:07.453748 1455 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:50:07.455263 1455 catalog_manager.cc:1347] Generated new cluster ID: 590e97120c3c46199e1ea24b9bbd7eef
I20260806 22:50:07.455314 1455 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 22:50:07 dist-test-slave-1xrh krb5kdc[1361](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056607, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.218.254@KRBTEST.COM
I20260806 22:50:07.462807 1459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:07.451957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37352 (local address 127.28.218.254:34595)
0806 22:50:07.452128 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:07.452137 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:07.452241 (+ 104us) server_negotiation.cc:407] Connection header received
0806 22:50:07.452373 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:07.452379 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:07.452446 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:07.452607 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:07.453473 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:07.454798 (+ 1325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:07.455838 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:07.456462 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:07.456574 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:07.459266 (+ 2692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:50:07.459270 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:50:07.459270 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:50:07.459281 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:50:07.461132 (+ 1851us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:07.461534 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:07.461537 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:07.461537 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:07.461627 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:07.461821 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:07.461821 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:07.461821 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:07.462095 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:50:07.462229 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:07.462348 (+ 119us) server_negotiation.cc:299] Negotiation successful
0806 22:50:07.462605 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":76,"threads_started":1}
I20260806 22:50:07.464341 1455 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:50:07.464844 1455 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:50:07.468132 1455 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c4a45111206a414eaae72449b43caeb0: Generated new TSK 0
I20260806 22:50:07.468290 1455 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.28.218.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.218.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.218.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.193.keytab.
Entry for principal kudu/127.28.218.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.193.keytab.
I20260806 22:50:07.497550 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056586788339-29547-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056586788339-29547-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056586788339-29547-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056586788339-29547-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.218.193:0
--local_ip_for_outbound_sockets=127.28.218.193
--webserver_interface=127.28.218.193
--webserver_port=0
--tserver_master_addrs=127.28.218.254:34595
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.193.keytab
--principal=kudu/127.28.218.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.218.212:46601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:50:07.578375 1472 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:50:07.578572 1472 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:50:07.578594 1472 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:50:07.579761 1472 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:50:07.579800 1472 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:50:07.579828 1472 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.218.193
I20260806 22:50:07.581228 1472 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:46601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056586788339-29547-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056586788339-29547-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.193.keytab
--principal=kudu/127.28.218.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.218.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056586788339-29547-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.218.193
--webserver_port=0
--tserver_master_addrs=127.28.218.254:34595
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1472
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.218.193
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056586788339-29547-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:50:07.581413 1472 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:50:07.581615 1472 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:50:07.586333 1477 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:07.586316 1480 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:50:07.586422 1472 server_base.cc:1034] running on GCE node
W20260806 22:50:07.586316 1478 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:50:07.586800 1472 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:50:07.587031 1472 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:50:07.588183 1472 hybrid_clock.cc:648] HybridClock initialized: now 1786056607588176 us; error 39 us; skew 500 ppm
Aug 06 22:50:07 dist-test-slave-1xrh krb5kdc[1361](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056607, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:50:07.590550 1472 init.cc:377] Logged in from keytab as kudu/127.28.218.193@KRBTEST.COM (short username kudu)
I20260806 22:50:07.590857 1472 fs_manager.cc:359] Metadata directory not provided
I20260806 22:50:07.590919 1472 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:50:07.591054 1472 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:50:07.592064 1472 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056586788339-29547-0/minicluster-data/ts-0/data/instance:
uuid: "89f290361bed4d6e83afd3250c4671c9"
format_stamp: "Formatted at 2026-08-06 22:50:07 on dist-test-slave-1xrh"
server_key: "3eb2d241fe435c5bc6a57b6f330402f4"
server_key_iv: "1214f584b9e3d84cf00dfb46e5d71d2c"
server_key_version: "encryptionkey@0"
I20260806 22:50:07.592404 1472 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056586788339-29547-0/minicluster-data/ts-0/wal/instance:
uuid: "89f290361bed4d6e83afd3250c4671c9"
format_stamp: "Formatted at 2026-08-06 22:50:07 on dist-test-slave-1xrh"
server_key: "3eb2d241fe435c5bc6a57b6f330402f4"
server_key_iv: "1214f584b9e3d84cf00dfb46e5d71d2c"
server_key_version: "encryptionkey@0"
I20260806 22:50:07.593638 1472 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:50:07.594292 1487 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:07.594451 1472 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:50:07.594551 1472 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056586788339-29547-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056586788339-29547-0/minicluster-data/ts-0/wal
uuid: "89f290361bed4d6e83afd3250c4671c9"
format_stamp: "Formatted at 2026-08-06 22:50:07 on dist-test-slave-1xrh"
server_key: "3eb2d241fe435c5bc6a57b6f330402f4"
server_key_iv: "1214f584b9e3d84cf00dfb46e5d71d2c"
server_key_version: "encryptionkey@0"
I20260806 22:50:07.594643 1472 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056586788339-29547-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056586788339-29547-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056586788339-29547-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:50:07.615203 1472 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:50:07.617161 1472 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:50:07.617304 1472 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:50:07.617538 1472 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:50:07.617833 1472 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:50:07.617882 1472 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:07.617940 1472 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:50:07.617974 1472 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:07.623862 1472 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.193:42531
I20260806 22:50:07.623904 1610 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.193:42531 every 8 connection(s)
I20260806 22:50:07.624161 1472 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056586788339-29547-0/minicluster-data/ts-0/data/info.pb
Aug 06 22:50:07 dist-test-slave-1xrh krb5kdc[1361](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056607, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.193@KRBTEST.COM for kudu/127.28.218.254@KRBTEST.COM
I20260806 22:50:07.632423 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 1472
I20260806 22:50:07.632540 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056586788339-29547-0/minicluster-data/ts-0/wal/instance
I20260806 22:50:07.632742 29547 external_mini_cluster.cc:1428] Setting key 1498f86bd4697671ec8f5145192e28de
I20260806 22:50:07.634087 1459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:07.624816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.193:53349 (local address 127.28.218.254:34595)
0806 22:50:07.624885 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:07.624887 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:07.625101 (+ 214us) server_negotiation.cc:407] Connection header received
0806 22:50:07.625704 (+ 603us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:07.625706 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:07.625723 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:07.625777 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:07.626829 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:07.627427 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:07.628429 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:07.628650 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:07.628720 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:07.630911 (+ 2191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:50:07.630912 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:50:07.630913 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:50:07.630920 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:50:07.632566 (+ 1646us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:07.633024 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:07.633026 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:07.633026 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:07.633109 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:07.633390 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:07.633390 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:07.633391 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:07.633528 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:50:07.633612 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:07.633890 (+ 278us) server_negotiation.cc:299] Negotiation successful
0806 22:50:07.633995 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 22:50:07.634254 1613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:07.624788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:34595 (local address 127.28.218.193:53349)
0806 22:50:07.624974 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:07.625010 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:07.625594 (+ 584us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:07.625852 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:07.625856 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:07.626158 (+ 302us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:07.626723 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:07.626728 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:07.627527 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:07.627528 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:07.628333 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:07.628335 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:07.628741 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:07.628742 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:07.628859 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:07.629370 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:50:07.629380 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:07.630818 (+ 1438us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:50:07.632672 (+ 1854us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:07.632678 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:07.632679 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:07.632949 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:07.633192 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:07.633193 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:07.633193 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:07.633314 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:07.633616 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:50:07.633623 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:50:07.633756 (+ 133us) client_negotiation.cc:769] Sending connection context
0806 22:50:07.633854 (+ 98us) client_negotiation.cc:240] Negotiation successful
0806 22:50:07.634008 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":89,"threads_started":1}
I20260806 22:50:07.634690 1611 heartbeater.cc:347] Connected to a master server at 127.28.218.254:34595
I20260806 22:50:07.634791 1611 heartbeater.cc:464] Registering TS with master...
I20260806 22:50:07.634943 1611 heartbeater.cc:511] Master 127.28.218.254:34595 requested a full tablet report, sending...
I20260806 22:50:07.635473 1407 ts_manager.cc:194] Registered new tserver with Master: 89f290361bed4d6e83afd3250c4671c9 (127.28.218.193:42531)
I20260806 22:50:07.636056 1407 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.218.193@KRBTEST.COM'} at 127.28.218.193:53349
WARNING: no policy specified for kudu/127.28.218.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.218.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.218.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.194.keytab.
Entry for principal kudu/127.28.218.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.194.keytab.
I20260806 22:50:07.657598 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056586788339-29547-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056586788339-29547-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056586788339-29547-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056586788339-29547-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.218.194:0
--local_ip_for_outbound_sockets=127.28.218.194
--webserver_interface=127.28.218.194
--webserver_port=0
--tserver_master_addrs=127.28.218.254:34595
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.194.keytab
--principal=kudu/127.28.218.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.218.212:46601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:50:07.738652 1616 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:50:07.738858 1616 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:50:07.738891 1616 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:50:07.740058 1616 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:50:07.740101 1616 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:50:07.740154 1616 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.218.194
I20260806 22:50:07.741631 1616 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:46601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056586788339-29547-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056586788339-29547-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.194.keytab
--principal=kudu/127.28.218.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.218.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056586788339-29547-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.218.194
--webserver_port=0
--tserver_master_addrs=127.28.218.254:34595
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1616
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.218.194
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056586788339-29547-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:50:07.741909 1616 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:50:07.742142 1616 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:50:07.746948 1625 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:07.746955 1624 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:07.746955 1628 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:50:07.747340 1616 server_base.cc:1034] running on GCE node
I20260806 22:50:07.747565 1616 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:50:07.747810 1616 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:50:07.748950 1616 hybrid_clock.cc:648] HybridClock initialized: now 1786056607748937 us; error 27 us; skew 500 ppm
Aug 06 22:50:07 dist-test-slave-1xrh krb5kdc[1361](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056607, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:50:07.751492 1616 init.cc:377] Logged in from keytab as kudu/127.28.218.194@KRBTEST.COM (short username kudu)
I20260806 22:50:07.751827 1616 fs_manager.cc:359] Metadata directory not provided
I20260806 22:50:07.751873 1616 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:50:07.752040 1616 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:50:07.753173 1616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056586788339-29547-0/minicluster-data/ts-1/data/instance:
uuid: "2d3b89d4880a4dfb875a5f8bab3d8063"
format_stamp: "Formatted at 2026-08-06 22:50:07 on dist-test-slave-1xrh"
server_key: "211665f6e04c659590926962989eccfe"
server_key_iv: "8f35ab3bf8df49a477e3e8fcf507cd82"
server_key_version: "encryptionkey@0"
I20260806 22:50:07.753523 1616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056586788339-29547-0/minicluster-data/ts-1/wal/instance:
uuid: "2d3b89d4880a4dfb875a5f8bab3d8063"
format_stamp: "Formatted at 2026-08-06 22:50:07 on dist-test-slave-1xrh"
server_key: "211665f6e04c659590926962989eccfe"
server_key_iv: "8f35ab3bf8df49a477e3e8fcf507cd82"
server_key_version: "encryptionkey@0"
I20260806 22:50:07.754688 1616 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:50:07.755389 1635 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:07.755545 1616 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:50:07.755661 1616 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056586788339-29547-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056586788339-29547-0/minicluster-data/ts-1/wal
uuid: "2d3b89d4880a4dfb875a5f8bab3d8063"
format_stamp: "Formatted at 2026-08-06 22:50:07 on dist-test-slave-1xrh"
server_key: "211665f6e04c659590926962989eccfe"
server_key_iv: "8f35ab3bf8df49a477e3e8fcf507cd82"
server_key_version: "encryptionkey@0"
I20260806 22:50:07.755728 1616 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056586788339-29547-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056586788339-29547-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056586788339-29547-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:50:07.771255 1616 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:50:07.773027 1616 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:50:07.773169 1616 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:50:07.773378 1616 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:50:07.773669 1616 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:50:07.773717 1616 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:07.773758 1616 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:50:07.773785 1616 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:07.779444 1616 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.194:39307
I20260806 22:50:07.779522 1759 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.194:39307 every 8 connection(s)
I20260806 22:50:07.779763 1616 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056586788339-29547-0/minicluster-data/ts-1/data/info.pb
I20260806 22:50:07.781925 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 1616
I20260806 22:50:07.782070 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056586788339-29547-0/minicluster-data/ts-1/wal/instance
I20260806 22:50:07.782270 29547 external_mini_cluster.cc:1428] Setting key 0b3c4fdcca664fbfbab84348b2b4e6d4
Aug 06 22:50:07 dist-test-slave-1xrh krb5kdc[1361](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056607, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.194@KRBTEST.COM for kudu/127.28.218.254@KRBTEST.COM
I20260806 22:50:07.789877 1459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:07.780425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.194:58801 (local address 127.28.218.254:34595)
0806 22:50:07.780475 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:07.780478 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:07.780699 (+ 221us) server_negotiation.cc:407] Connection header received
0806 22:50:07.781138 (+ 439us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:07.781139 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:07.781152 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:07.781219 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:07.782158 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:07.782764 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:07.783927 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:07.784272 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:07.784369 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:07.786905 (+ 2536us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:50:07.786911 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:50:07.786911 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:50:07.786917 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:50:07.788279 (+ 1362us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:07.788859 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:07.788860 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:07.788860 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:07.788923 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:07.789235 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:07.789235 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:07.789235 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:07.789332 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:50:07.789380 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:07.789748 (+ 368us) server_negotiation.cc:299] Negotiation successful
0806 22:50:07.789812 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 22:50:07.790225 1762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:07.780390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:34595 (local address 127.28.218.194:58801)
0806 22:50:07.780544 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:07.780584 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:07.781045 (+ 461us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:07.781273 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:07.781276 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:07.781535 (+ 259us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:07.781998 (+ 463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:07.782002 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:07.782896 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:07.782897 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:07.783828 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:07.783831 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:07.784371 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:07.784372 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:07.784476 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:07.784960 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:50:07.784969 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:07.786798 (+ 1829us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:50:07.788367 (+ 1569us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:07.788375 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:07.788378 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:07.788749 (+ 371us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:07.788992 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:07.788993 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:07.788994 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:07.789160 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:07.789399 (+ 239us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:50:07.789406 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:50:07.789595 (+ 189us) client_negotiation.cc:769] Sending connection context
0806 22:50:07.789736 (+ 141us) client_negotiation.cc:240] Negotiation successful
0806 22:50:07.789936 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":89,"threads_started":1}
I20260806 22:50:07.790753 1760 heartbeater.cc:347] Connected to a master server at 127.28.218.254:34595
I20260806 22:50:07.790838 1760 heartbeater.cc:464] Registering TS with master...
I20260806 22:50:07.790997 1760 heartbeater.cc:511] Master 127.28.218.254:34595 requested a full tablet report, sending...
I20260806 22:50:07.791401 1407 ts_manager.cc:194] Registered new tserver with Master: 2d3b89d4880a4dfb875a5f8bab3d8063 (127.28.218.194:39307)
I20260806 22:50:07.791728 1407 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.218.194@KRBTEST.COM'} at 127.28.218.194:58801
WARNING: no policy specified for kudu/127.28.218.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.218.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.218.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.195.keytab.
Entry for principal kudu/127.28.218.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.195.keytab.
I20260806 22:50:07.810016 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056586788339-29547-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056586788339-29547-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056586788339-29547-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056586788339-29547-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.218.195:0
--local_ip_for_outbound_sockets=127.28.218.195
--webserver_interface=127.28.218.195
--webserver_port=0
--tserver_master_addrs=127.28.218.254:34595
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.195.keytab
--principal=kudu/127.28.218.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.218.212:46601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:50:07.889616 1765 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:50:07.889829 1765 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:50:07.889865 1765 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:50:07.890992 1765 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:50:07.891053 1765 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:50:07.891098 1765 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.218.195
I20260806 22:50:07.892563 1765 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:46601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056586788339-29547-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056586788339-29547-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.195.keytab
--principal=kudu/127.28.218.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.218.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056586788339-29547-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.218.195
--webserver_port=0
--tserver_master_addrs=127.28.218.254:34595
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1765
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.218.195
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056586788339-29547-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:50:07.892843 1765 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:50:07.893056 1765 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:50:07.898028 1771 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:07.898079 1774 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:07.898036 1770 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:50:07.898237 1765 server_base.cc:1034] running on GCE node
I20260806 22:50:07.898480 1765 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:50:07.898665 1765 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:50:07.899786 1765 hybrid_clock.cc:648] HybridClock initialized: now 1786056607899778 us; error 23 us; skew 500 ppm
Aug 06 22:50:07 dist-test-slave-1xrh krb5kdc[1361](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056607, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:50:07.902074 1765 init.cc:377] Logged in from keytab as kudu/127.28.218.195@KRBTEST.COM (short username kudu)
I20260806 22:50:07.902379 1765 fs_manager.cc:359] Metadata directory not provided
I20260806 22:50:07.902422 1765 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:50:07.902580 1765 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:50:07.903587 1765 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056586788339-29547-0/minicluster-data/ts-2/data/instance:
uuid: "a52a8badd56f47fc9833529a2573dd5b"
format_stamp: "Formatted at 2026-08-06 22:50:07 on dist-test-slave-1xrh"
server_key: "4aed4de409e9755947f0b3b4c7938149"
server_key_iv: "57772399d17689517ee70230c861c5d2"
server_key_version: "encryptionkey@0"
I20260806 22:50:07.903927 1765 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056586788339-29547-0/minicluster-data/ts-2/wal/instance:
uuid: "a52a8badd56f47fc9833529a2573dd5b"
format_stamp: "Formatted at 2026-08-06 22:50:07 on dist-test-slave-1xrh"
server_key: "4aed4de409e9755947f0b3b4c7938149"
server_key_iv: "57772399d17689517ee70230c861c5d2"
server_key_version: "encryptionkey@0"
I20260806 22:50:07.905148 1765 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:50:07.905814 1781 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:07.905982 1765 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:50:07.906057 1765 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056586788339-29547-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056586788339-29547-0/minicluster-data/ts-2/wal
uuid: "a52a8badd56f47fc9833529a2573dd5b"
format_stamp: "Formatted at 2026-08-06 22:50:07 on dist-test-slave-1xrh"
server_key: "4aed4de409e9755947f0b3b4c7938149"
server_key_iv: "57772399d17689517ee70230c861c5d2"
server_key_version: "encryptionkey@0"
I20260806 22:50:07.906107 1765 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056586788339-29547-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056586788339-29547-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056586788339-29547-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:50:07.915004 1765 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:50:07.917507 1765 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:50:07.917644 1765 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:50:07.917878 1765 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:50:07.918272 1765 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:50:07.918322 1765 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:07.918359 1765 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:50:07.918388 1765 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:07.924391 1765 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.195:35743
I20260806 22:50:07.924451 1901 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.195:35743 every 8 connection(s)
I20260806 22:50:07.924705 1765 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056586788339-29547-0/minicluster-data/ts-2/data/info.pb
Aug 06 22:50:07 dist-test-slave-1xrh krb5kdc[1361](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056607, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.195@KRBTEST.COM for kudu/127.28.218.254@KRBTEST.COM
I20260806 22:50:07.933153 1459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:07.925488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.195:36007 (local address 127.28.218.254:34595)
0806 22:50:07.925547 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:07.925549 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:07.925746 (+ 197us) server_negotiation.cc:407] Connection header received
0806 22:50:07.926218 (+ 472us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:07.926219 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:07.926229 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:07.926281 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:07.927115 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:07.927653 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:07.928637 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:07.928819 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:07.928869 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:07.930879 (+ 2010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:50:07.930880 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:50:07.930881 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:50:07.930886 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:50:07.932093 (+ 1207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:07.932434 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:07.932435 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:07.932435 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:07.932492 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:07.932683 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:07.932684 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:07.932684 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:07.932777 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:50:07.932824 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:07.933033 (+ 209us) server_negotiation.cc:299] Negotiation successful
0806 22:50:07.933091 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 22:50:07.933305 1904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:07.925476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:34595 (local address 127.28.218.195:36007)
0806 22:50:07.925631 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:07.925664 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:07.926135 (+ 471us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:07.926330 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:07.926333 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:07.926575 (+ 242us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:07.927040 (+ 465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:07.927046 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:07.927737 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:07.927738 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:07.928561 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:07.928564 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:07.928891 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:07.928892 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:07.928995 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:07.929476 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:50:07.929484 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:07.930796 (+ 1312us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:50:07.932160 (+ 1364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:07.932165 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:07.932167 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:07.932371 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:07.932544 (+ 173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:07.932544 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:07.932545 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:07.932634 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:07.932825 (+ 191us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:50:07.932829 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:50:07.932932 (+ 103us) client_negotiation.cc:769] Sending connection context
0806 22:50:07.933008 (+ 76us) client_negotiation.cc:240] Negotiation successful
0806 22:50:07.933119 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":89,"threads_started":1}
I20260806 22:50:07.933760 1902 heartbeater.cc:347] Connected to a master server at 127.28.218.254:34595
I20260806 22:50:07.933857 1902 heartbeater.cc:464] Registering TS with master...
I20260806 22:50:07.934000 1902 heartbeater.cc:511] Master 127.28.218.254:34595 requested a full tablet report, sending...
I20260806 22:50:07.934365 1407 ts_manager.cc:194] Registered new tserver with Master: a52a8badd56f47fc9833529a2573dd5b (127.28.218.195:35743)
I20260806 22:50:07.934478 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 1765
I20260806 22:50:07.934588 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056586788339-29547-0/minicluster-data/ts-2/wal/instance
I20260806 22:50:07.934780 29547 external_mini_cluster.cc:1428] Setting key 60c767ce23c35f736dda999eedb9ab63
I20260806 22:50:07.934873 1407 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.218.195@KRBTEST.COM'} at 127.28.218.195:36007
I20260806 22:50:07.935706 29547 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:50:07.942080 1459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:07.936948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37364 (local address 127.28.218.254:34595)
0806 22:50:07.937001 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:07.937003 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:07.937091 (+ 88us) server_negotiation.cc:407] Connection header received
0806 22:50:07.937130 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:07.937131 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:07.937138 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:07.937183 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:07.937677 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:07.938110 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:07.938748 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:07.938921 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:07.938966 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:07.939967 (+ 1001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:50:07.939969 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:50:07.939969 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:50:07.939974 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:50:07.941269 (+ 1295us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:07.941545 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:07.941546 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:07.941546 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:07.941591 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:07.941725 (+ 134us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:07.941726 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:07.941726 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:07.941831 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:50:07.941897 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:07.941973 (+ 76us) server_negotiation.cc:299] Negotiation successful
0806 22:50:07.942025 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 22:50:07.943375 1407 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37364:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:50:07.944890 1407 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:50:07.950987 1915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:07.946994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37016 (local address 127.28.218.195:35743)
0806 22:50:07.947199 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:07.947203 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:07.947215 (+ 12us) server_negotiation.cc:407] Connection header received
0806 22:50:07.947262 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:07.947264 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:07.947281 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:07.947373 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:50:07.947620 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:07.948691 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:07.949898 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:07.950606 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:07.950718 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:07.950773 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:07.950904 (+ 131us) server_negotiation.cc:299] Negotiation successful
0806 22:50:07.950934 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":65,"threads_started":1}
I20260806 22:50:07.951037 1912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:07.946816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.195:35743 (local address 127.0.0.1:37016)
0806 22:50:07.947168 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:07.947176 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:07.947209 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:07.947390 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:07.947393 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:07.947396 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:50:07.947461 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:07.947462 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:07.948791 (+ 1329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:07.948792 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:07.949816 (+ 1024us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:07.949817 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:07.950735 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:07.950736 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:07.950811 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:07.950824 (+ 13us) client_negotiation.cc:769] Sending connection context
0806 22:50:07.950896 (+ 72us) client_negotiation.cc:240] Negotiation successful
0806 22:50:07.950947 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":333,"mutex_wait_us":12,"thread_start_us":64,"threads_started":1}
I20260806 22:50:07.951584 1913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:07.946882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43452 (local address 127.28.218.194:39307)
0806 22:50:07.947305 (+ 423us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:07.947309 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:07.947316 (+ 7us) server_negotiation.cc:407] Connection header received
0806 22:50:07.947363 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:07.947365 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:07.947382 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:07.947575 (+ 193us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:50:07.947656 (+ 81us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:07.948469 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:07.949898 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:07.950606 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:07.950719 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:07.950773 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:07.951459 (+ 686us) server_negotiation.cc:299] Negotiation successful
0806 22:50:07.951504 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":397,"thread_start_us":92,"threads_started":1}
I20260806 22:50:07.951869 1910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:07.946815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.193:42531 (local address 127.0.0.1:47372)
0806 22:50:07.947050 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:07.947060 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:07.947100 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:07.947486 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:07.947487 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:07.947489 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:50:07.947541 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:07.947542 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:07.949902 (+ 2360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:07.949903 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:07.950557 (+ 654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:07.950559 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:07.951614 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:07.951615 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:07.951720 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:07.951724 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:50:07.951774 (+ 50us) client_negotiation.cc:240] Negotiation successful
0806 22:50:07.951808 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":62,"threads_started":1}
I20260806 22:50:07.951924 1831 tablet_service.cc:1467] Processing CreateTablet for tablet 26cfc5f2554c401a9aa92e4ff5ddde90 (DEFAULT_TABLE table=test-table [id=6af460fcab13486baaec9f457afb94ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:50:07.952189 1916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:07.947051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47372 (local address 127.28.218.193:42531)
0806 22:50:07.947264 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:07.947267 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:07.947276 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:50:07.947314 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:07.947315 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:07.947329 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:07.947400 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:50:07.947624 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:07.948693 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:07.950648 (+ 1955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:07.951470 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:07.951531 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:07.951582 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:07.951971 (+ 389us) server_negotiation.cc:299] Negotiation successful
0806 22:50:07.952113 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":105,"threads_started":1}
I20260806 22:50:07.952293 1831 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26cfc5f2554c401a9aa92e4ff5ddde90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:50:07.952752 1540 tablet_service.cc:1467] Processing CreateTablet for tablet 26cfc5f2554c401a9aa92e4ff5ddde90 (DEFAULT_TABLE table=test-table [id=6af460fcab13486baaec9f457afb94ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:50:07.953069 1540 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26cfc5f2554c401a9aa92e4ff5ddde90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:50:07.954877 1917 tablet_bootstrap.cc:492] T 26cfc5f2554c401a9aa92e4ff5ddde90 P a52a8badd56f47fc9833529a2573dd5b: Bootstrap starting.
I20260806 22:50:07.955230 1911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:07.946791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.194:39307 (local address 127.0.0.1:43452)
0806 22:50:07.946942 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:07.946978 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:07.947040 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:07.947510 (+ 470us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:07.947510 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:07.947512 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:50:07.947541 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:07.947542 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:07.948561 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:07.948562 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:07.949804 (+ 1242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:07.949807 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:07.951278 (+ 1471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:07.951279 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:07.951376 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:07.951379 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:50:07.951440 (+ 61us) client_negotiation.cc:240] Negotiation successful
0806 22:50:07.951495 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":52,"threads_started":1}
I20260806 22:50:07.955571 1917 tablet_bootstrap.cc:654] T 26cfc5f2554c401a9aa92e4ff5ddde90 P a52a8badd56f47fc9833529a2573dd5b: Neither blocks nor log segments found. Creating new log.
I20260806 22:50:07.955961 1686 tablet_service.cc:1467] Processing CreateTablet for tablet 26cfc5f2554c401a9aa92e4ff5ddde90 (DEFAULT_TABLE table=test-table [id=6af460fcab13486baaec9f457afb94ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:50:07.956236 1686 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26cfc5f2554c401a9aa92e4ff5ddde90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:50:07.956394 1917 log.cc:824] T 26cfc5f2554c401a9aa92e4ff5ddde90 P a52a8badd56f47fc9833529a2573dd5b: Log is configured to *not* fsync() on all Append() calls
I20260806 22:50:07.956739 1918 tablet_bootstrap.cc:492] T 26cfc5f2554c401a9aa92e4ff5ddde90 P 89f290361bed4d6e83afd3250c4671c9: Bootstrap starting.
I20260806 22:50:07.957095 1917 tablet_bootstrap.cc:492] T 26cfc5f2554c401a9aa92e4ff5ddde90 P a52a8badd56f47fc9833529a2573dd5b: No bootstrap required, opened a new log
I20260806 22:50:07.957146 1917 ts_tablet_manager.cc:1397] T 26cfc5f2554c401a9aa92e4ff5ddde90 P a52a8badd56f47fc9833529a2573dd5b: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:50:07.958674 1917 raft_consensus.cc:348] T 26cfc5f2554c401a9aa92e4ff5ddde90 P a52a8badd56f47fc9833529a2573dd5b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d3b89d4880a4dfb875a5f8bab3d8063" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 39307 } } peers { permanent_uuid: "a52a8badd56f47fc9833529a2573dd5b" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 35743 } } peers { permanent_uuid: "89f290361bed4d6e83afd3250c4671c9" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 42531 } }
I20260806 22:50:07.958817 1917 raft_consensus.cc:374] T 26cfc5f2554c401a9aa92e4ff5ddde90 P a52a8badd56f47fc9833529a2573dd5b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:50:07.958863 1917 raft_consensus.cc:729] T 26cfc5f2554c401a9aa92e4ff5ddde90 P a52a8badd56f47fc9833529a2573dd5b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a52a8badd56f47fc9833529a2573dd5b, State: Initialized, Role: FOLLOWER
I20260806 22:50:07.959013 1920 tablet_bootstrap.cc:492] T 26cfc5f2554c401a9aa92e4ff5ddde90 P 2d3b89d4880a4dfb875a5f8bab3d8063: Bootstrap starting.
I20260806 22:50:07.958985 1917 consensus_queue.cc:260] T 26cfc5f2554c401a9aa92e4ff5ddde90 P a52a8badd56f47fc9833529a2573dd5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d3b89d4880a4dfb875a5f8bab3d8063" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 39307 } } peers { permanent_uuid: "a52a8badd56f47fc9833529a2573dd5b" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 35743 } } peers { permanent_uuid: "89f290361bed4d6e83afd3250c4671c9" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 42531 } }
I20260806 22:50:07.959304 1917 ts_tablet_manager.cc:1428] T 26cfc5f2554c401a9aa92e4ff5ddde90 P a52a8badd56f47fc9833529a2573dd5b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:50:07.959309 1902 heartbeater.cc:503] Master 127.28.218.254:34595 was elected leader, sending a full tablet report...
I20260806 22:50:07.959649 1920 tablet_bootstrap.cc:654] T 26cfc5f2554c401a9aa92e4ff5ddde90 P 2d3b89d4880a4dfb875a5f8bab3d8063: Neither blocks nor log segments found. Creating new log.
I20260806 22:50:07.959849 1918 tablet_bootstrap.cc:654] T 26cfc5f2554c401a9aa92e4ff5ddde90 P 89f290361bed4d6e83afd3250c4671c9: Neither blocks nor log segments found. Creating new log.
I20260806 22:50:07.960268 1920 log.cc:824] T 26cfc5f2554c401a9aa92e4ff5ddde90 P 2d3b89d4880a4dfb875a5f8bab3d8063: Log is configured to *not* fsync() on all Append() calls
I20260806 22:50:07.960471 1918 log.cc:824] T 26cfc5f2554c401a9aa92e4ff5ddde90 P 89f290361bed4d6e83afd3250c4671c9: Log is configured to *not* fsync() on all Append() calls
I20260806 22:50:07.961067 1920 tablet_bootstrap.cc:492] T 26cfc5f2554c401a9aa92e4ff5ddde90 P 2d3b89d4880a4dfb875a5f8bab3d8063: No bootstrap required, opened a new log
I20260806 22:50:07.961140 1920 ts_tablet_manager.cc:1397] T 26cfc5f2554c401a9aa92e4ff5ddde90 P 2d3b89d4880a4dfb875a5f8bab3d8063: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:50:07.961227 1918 tablet_bootstrap.cc:492] T 26cfc5f2554c401a9aa92e4ff5ddde90 P 89f290361bed4d6e83afd3250c4671c9: No bootstrap required, opened a new log
I20260806 22:50:07.961289 1918 ts_tablet_manager.cc:1397] T 26cfc5f2554c401a9aa92e4ff5ddde90 P 89f290361bed4d6e83afd3250c4671c9: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260806 22:50:07.963109 1920 raft_consensus.cc:348] T 26cfc5f2554c401a9aa92e4ff5ddde90 P 2d3b89d4880a4dfb875a5f8bab3d8063 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d3b89d4880a4dfb875a5f8bab3d8063" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 39307 } } peers { permanent_uuid: "a52a8badd56f47fc9833529a2573dd5b" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 35743 } } peers { permanent_uuid: "89f290361bed4d6e83afd3250c4671c9" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 42531 } }
I20260806 22:50:07.963245 1918 raft_consensus.cc:348] T 26cfc5f2554c401a9aa92e4ff5ddde90 P 89f290361bed4d6e83afd3250c4671c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d3b89d4880a4dfb875a5f8bab3d8063" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 39307 } } peers { permanent_uuid: "a52a8badd56f47fc9833529a2573dd5b" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 35743 } } peers { permanent_uuid: "89f290361bed4d6e83afd3250c4671c9" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 42531 } }
I20260806 22:50:07.963337 1920 raft_consensus.cc:374] T 26cfc5f2554c401a9aa92e4ff5ddde90 P 2d3b89d4880a4dfb875a5f8bab3d8063 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:50:07.963374 1918 raft_consensus.cc:374] T 26cfc5f2554c401a9aa92e4ff5ddde90 P 89f290361bed4d6e83afd3250c4671c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:50:07.963373 1920 raft_consensus.cc:729] T 26cfc5f2554c401a9aa92e4ff5ddde90 P 2d3b89d4880a4dfb875a5f8bab3d8063 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d3b89d4880a4dfb875a5f8bab3d8063, State: Initialized, Role: FOLLOWER
I20260806 22:50:07.963428 1918 raft_consensus.cc:729] T 26cfc5f2554c401a9aa92e4ff5ddde90 P 89f290361bed4d6e83afd3250c4671c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89f290361bed4d6e83afd3250c4671c9, State: Initialized, Role: FOLLOWER
I20260806 22:50:07.963559 1920 consensus_queue.cc:260] T 26cfc5f2554c401a9aa92e4ff5ddde90 P 2d3b89d4880a4dfb875a5f8bab3d8063 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d3b89d4880a4dfb875a5f8bab3d8063" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 39307 } } peers { permanent_uuid: "a52a8badd56f47fc9833529a2573dd5b" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 35743 } } peers { permanent_uuid: "89f290361bed4d6e83afd3250c4671c9" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 42531 } }
I20260806 22:50:07.963559 1918 consensus_queue.cc:260] T 26cfc5f2554c401a9aa92e4ff5ddde90 P 89f290361bed4d6e83afd3250c4671c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d3b89d4880a4dfb875a5f8bab3d8063" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 39307 } } peers { permanent_uuid: "a52a8badd56f47fc9833529a2573dd5b" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 35743 } } peers { permanent_uuid: "89f290361bed4d6e83afd3250c4671c9" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 42531 } }
I20260806 22:50:07.963863 1920 ts_tablet_manager.cc:1428] T 26cfc5f2554c401a9aa92e4ff5ddde90 P 2d3b89d4880a4dfb875a5f8bab3d8063: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260806 22:50:07.963863 1918 ts_tablet_manager.cc:1428] T 26cfc5f2554c401a9aa92e4ff5ddde90 P 89f290361bed4d6e83afd3250c4671c9: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 22:50:07.963876 1611 heartbeater.cc:503] Master 127.28.218.254:34595 was elected leader, sending a full tablet report...
I20260806 22:50:07.964021 1760 heartbeater.cc:503] Master 127.28.218.254:34595 was elected leader, sending a full tablet report...
I20260806 22:50:08.023797 1922 raft_consensus.cc:482] T 26cfc5f2554c401a9aa92e4ff5ddde90 P a52a8badd56f47fc9833529a2573dd5b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:50:08.023981 1922 raft_consensus.cc:504] T 26cfc5f2554c401a9aa92e4ff5ddde90 P a52a8badd56f47fc9833529a2573dd5b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d3b89d4880a4dfb875a5f8bab3d8063" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 39307 } } peers { permanent_uuid: "a52a8badd56f47fc9833529a2573dd5b" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 35743 } } peers { permanent_uuid: "89f290361bed4d6e83afd3250c4671c9" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 42531 } }
I20260806 22:50:08.024359 1922 leader_election.cc:288] T 26cfc5f2554c401a9aa92e4ff5ddde90 P a52a8badd56f47fc9833529a2573dd5b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2d3b89d4880a4dfb875a5f8bab3d8063 (127.28.218.194:39307), 89f290361bed4d6e83afd3250c4671c9 (127.28.218.193:42531)
I20260806 22:50:08.028267 1916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:08.024631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.195:54737 (local address 127.28.218.193:42531)
0806 22:50:08.024675 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:08.024677 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:08.024804 (+ 127us) server_negotiation.cc:407] Connection header received
0806 22:50:08.024849 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:08.024849 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:08.024856 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:08.024893 (+ 37us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:50:08.025045 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:08.025737 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:08.027327 (+ 1590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:08.027925 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:08.027999 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:08.028008 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:08.028159 (+ 151us) server_negotiation.cc:299] Negotiation successful
0806 22:50:08.028196 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 22:50:08.028267 1913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:08.024587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.195:48449 (local address 127.28.218.194:39307)
0806 22:50:08.024646 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:08.024648 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:08.024657 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:50:08.024686 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:08.024687 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:08.024697 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:08.024756 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:50:08.024956 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:08.025707 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:08.027345 (+ 1638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:08.027925 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:08.028004 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:08.028011 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:08.028159 (+ 148us) server_negotiation.cc:299] Negotiation successful
0806 22:50:08.028196 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 22:50:08.028285 1904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:08.024500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.194:39307 (local address 127.28.218.195:48449)
0806 22:50:08.024556 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:08.024569 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:08.024619 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:08.024783 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:08.024784 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:08.024787 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:50:08.024864 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:08.024866 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:08.025809 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:08.025810 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:08.027244 (+ 1434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:08.027248 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:08.028007 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:08.028008 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:08.028091 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:08.028093 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 22:50:08.028146 (+ 53us) client_negotiation.cc:240] Negotiation successful
0806 22:50:08.028186 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33,"mutex_wait_us":117}
I20260806 22:50:08.028303 1927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:08.024525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.193:42531 (local address 127.28.218.195:54737)
0806 22:50:08.024716 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:08.024724 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:08.024788 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:08.024911 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:08.024912 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:08.024914 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:50:08.024978 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:08.024983 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:08.025809 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:08.025810 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:08.027244 (+ 1434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:08.027248 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:08.028011 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:08.028011 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:08.028081 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:08.028084 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:50:08.028141 (+ 57us) client_negotiation.cc:240] Negotiation successful
0806 22:50:08.028186 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"mutex_wait_us":65,"thread_start_us":125,"threads_started":1}
I20260806 22:50:08.028748 1560 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26cfc5f2554c401a9aa92e4ff5ddde90" candidate_uuid: "a52a8badd56f47fc9833529a2573dd5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89f290361bed4d6e83afd3250c4671c9" is_pre_election: true
I20260806 22:50:08.028743 1708 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26cfc5f2554c401a9aa92e4ff5ddde90" candidate_uuid: "a52a8badd56f47fc9833529a2573dd5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d3b89d4880a4dfb875a5f8bab3d8063" is_pre_election: true
I20260806 22:50:08.028900 1708 raft_consensus.cc:2454] T 26cfc5f2554c401a9aa92e4ff5ddde90 P 2d3b89d4880a4dfb875a5f8bab3d8063 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a52a8badd56f47fc9833529a2573dd5b in term 0.
I20260806 22:50:08.028900 1560 raft_consensus.cc:2454] T 26cfc5f2554c401a9aa92e4ff5ddde90 P 89f290361bed4d6e83afd3250c4671c9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a52a8badd56f47fc9833529a2573dd5b in term 0.
I20260806 22:50:08.029150 1785 leader_election.cc:302] T 26cfc5f2554c401a9aa92e4ff5ddde90 P a52a8badd56f47fc9833529a2573dd5b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d3b89d4880a4dfb875a5f8bab3d8063, a52a8badd56f47fc9833529a2573dd5b; no voters:
I20260806 22:50:08.029280 1922 raft_consensus.cc:2781] T 26cfc5f2554c401a9aa92e4ff5ddde90 P a52a8badd56f47fc9833529a2573dd5b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:50:08.029337 1922 raft_consensus.cc:482] T 26cfc5f2554c401a9aa92e4ff5ddde90 P a52a8badd56f47fc9833529a2573dd5b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:50:08.029361 1922 raft_consensus.cc:3037] T 26cfc5f2554c401a9aa92e4ff5ddde90 P a52a8badd56f47fc9833529a2573dd5b [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:50:08.030014 1922 raft_consensus.cc:504] T 26cfc5f2554c401a9aa92e4ff5ddde90 P a52a8badd56f47fc9833529a2573dd5b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d3b89d4880a4dfb875a5f8bab3d8063" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 39307 } } peers { permanent_uuid: "a52a8badd56f47fc9833529a2573dd5b" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 35743 } } peers { permanent_uuid: "89f290361bed4d6e83afd3250c4671c9" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 42531 } }
I20260806 22:50:08.030140 1922 leader_election.cc:288] T 26cfc5f2554c401a9aa92e4ff5ddde90 P a52a8badd56f47fc9833529a2573dd5b [CANDIDATE]: Term 1 election: Requested vote from peers 2d3b89d4880a4dfb875a5f8bab3d8063 (127.28.218.194:39307), 89f290361bed4d6e83afd3250c4671c9 (127.28.218.193:42531)
I20260806 22:50:08.030272 1708 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26cfc5f2554c401a9aa92e4ff5ddde90" candidate_uuid: "a52a8badd56f47fc9833529a2573dd5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d3b89d4880a4dfb875a5f8bab3d8063"
W20260806 22:50:08.030311 1761 tablet.cc:2369] T 26cfc5f2554c401a9aa92e4ff5ddde90 P 2d3b89d4880a4dfb875a5f8bab3d8063: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:50:08.030289 1560 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26cfc5f2554c401a9aa92e4ff5ddde90" candidate_uuid: "a52a8badd56f47fc9833529a2573dd5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89f290361bed4d6e83afd3250c4671c9"
I20260806 22:50:08.030350 1708 raft_consensus.cc:3037] T 26cfc5f2554c401a9aa92e4ff5ddde90 P 2d3b89d4880a4dfb875a5f8bab3d8063 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:50:08.030360 1560 raft_consensus.cc:3037] T 26cfc5f2554c401a9aa92e4ff5ddde90 P 89f290361bed4d6e83afd3250c4671c9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:50:08.031064 1560 raft_consensus.cc:2454] T 26cfc5f2554c401a9aa92e4ff5ddde90 P 89f290361bed4d6e83afd3250c4671c9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a52a8badd56f47fc9833529a2573dd5b in term 1.
I20260806 22:50:08.031068 1708 raft_consensus.cc:2454] T 26cfc5f2554c401a9aa92e4ff5ddde90 P 2d3b89d4880a4dfb875a5f8bab3d8063 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a52a8badd56f47fc9833529a2573dd5b in term 1.
I20260806 22:50:08.031239 1784 leader_election.cc:302] T 26cfc5f2554c401a9aa92e4ff5ddde90 P a52a8badd56f47fc9833529a2573dd5b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 89f290361bed4d6e83afd3250c4671c9, a52a8badd56f47fc9833529a2573dd5b; no voters:
I20260806 22:50:08.031350 1922 raft_consensus.cc:2781] T 26cfc5f2554c401a9aa92e4ff5ddde90 P a52a8badd56f47fc9833529a2573dd5b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:50:08.031515 1922 raft_consensus.cc:686] T 26cfc5f2554c401a9aa92e4ff5ddde90 P a52a8badd56f47fc9833529a2573dd5b [term 1 LEADER]: Becoming Leader. State: Replica: a52a8badd56f47fc9833529a2573dd5b, State: Running, Role: LEADER
I20260806 22:50:08.031618 1922 consensus_queue.cc:237] T 26cfc5f2554c401a9aa92e4ff5ddde90 P a52a8badd56f47fc9833529a2573dd5b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d3b89d4880a4dfb875a5f8bab3d8063" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 39307 } } peers { permanent_uuid: "a52a8badd56f47fc9833529a2573dd5b" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 35743 } } peers { permanent_uuid: "89f290361bed4d6e83afd3250c4671c9" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 42531 } }
I20260806 22:50:08.032483 1406 catalog_manager.cc:5495] T 26cfc5f2554c401a9aa92e4ff5ddde90 P a52a8badd56f47fc9833529a2573dd5b reported cstate change: term changed from 0 to 1, leader changed from <none> to a52a8badd56f47fc9833529a2573dd5b (127.28.218.195). New cstate: current_term: 1 leader_uuid: "a52a8badd56f47fc9833529a2573dd5b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d3b89d4880a4dfb875a5f8bab3d8063" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 39307 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a52a8badd56f47fc9833529a2573dd5b" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 35743 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "89f290361bed4d6e83afd3250c4671c9" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 42531 } health_report { overall_health: UNKNOWN } } }
I20260806 22:50:08.041113 1915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:08.039225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37030 (local address 127.28.218.195:35743)
0806 22:50:08.039261 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:08.039263 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:08.039269 (+ 6us) server_negotiation.cc:407] Connection header received
0806 22:50:08.039292 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:08.039292 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:08.039306 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:08.039363 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:50:08.039474 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:08.039883 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:08.040548 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:08.040716 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:08.040762 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:08.040885 (+ 123us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:50:08.040964 (+ 79us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:50:08.041021 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:08.041053 (+ 32us) server_negotiation.cc:299] Negotiation successful
0806 22:50:08.041068 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 22:50:08.042594 1560 raft_consensus.cc:1261] T 26cfc5f2554c401a9aa92e4ff5ddde90 P 89f290361bed4d6e83afd3250c4671c9 [term 1 FOLLOWER]: Refusing update from remote peer a52a8badd56f47fc9833529a2573dd5b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:50:08.042594 1708 raft_consensus.cc:1261] T 26cfc5f2554c401a9aa92e4ff5ddde90 P 2d3b89d4880a4dfb875a5f8bab3d8063 [term 1 FOLLOWER]: Refusing update from remote peer a52a8badd56f47fc9833529a2573dd5b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:50:08.042882 1922 consensus_queue.cc:1035] T 26cfc5f2554c401a9aa92e4ff5ddde90 P a52a8badd56f47fc9833529a2573dd5b [LEADER]: Connected to new peer: Peer: permanent_uuid: "89f290361bed4d6e83afd3250c4671c9" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:50:08.043015 1928 consensus_queue.cc:1035] T 26cfc5f2554c401a9aa92e4ff5ddde90 P a52a8badd56f47fc9833529a2573dd5b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d3b89d4880a4dfb875a5f8bab3d8063" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 39307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:50:08.045147 1931 mvcc.cc:205] Tried to move back new op lower bound from 7315687866540437504 to 7315687866498248704. Current Snapshot: MvccSnapshot[applied={T|T < 7315687866540437504}]
I20260806 22:50:08.045310 1932 mvcc.cc:205] Tried to move back new op lower bound from 7315687866540437504 to 7315687866498248704. Current Snapshot: MvccSnapshot[applied={T|T < 7315687866540437504}]
I20260806 22:50:08.045389 1933 mvcc.cc:205] Tried to move back new op lower bound from 7315687866540437504 to 7315687866498248704. Current Snapshot: MvccSnapshot[applied={T|T < 7315687866540437504}]
I20260806 22:50:08.047685 1406 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37364:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:50:08.047765 1406 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37364:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:50:08.048476 1406 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c4a45111206a414eaae72449b43caeb0: Sending DeleteTablet for 3 replicas of tablet 26cfc5f2554c401a9aa92e4ff5ddde90
I20260806 22:50:08.048772 1686 tablet_service.cc:1514] Processing DeleteTablet for tablet 26cfc5f2554c401a9aa92e4ff5ddde90 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:50:08 UTC) from {username='kudu', principal='kudu/127.28.218.254@KRBTEST.COM'} at 127.0.0.1:43452
I20260806 22:50:08.048777 1831 tablet_service.cc:1514] Processing DeleteTablet for tablet 26cfc5f2554c401a9aa92e4ff5ddde90 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:50:08 UTC) from {username='kudu', principal='kudu/127.28.218.254@KRBTEST.COM'} at 127.0.0.1:37016
I20260806 22:50:08.048846 1540 tablet_service.cc:1514] Processing DeleteTablet for tablet 26cfc5f2554c401a9aa92e4ff5ddde90 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:50:08 UTC) from {username='kudu', principal='kudu/127.28.218.254@KRBTEST.COM'} at 127.0.0.1:47372
I20260806 22:50:08.049016 1941 tablet_replica.cc:330] T 26cfc5f2554c401a9aa92e4ff5ddde90 P a52a8badd56f47fc9833529a2573dd5b: stopping tablet replica
I20260806 22:50:08.049049 1943 tablet_replica.cc:330] T 26cfc5f2554c401a9aa92e4ff5ddde90 P 89f290361bed4d6e83afd3250c4671c9: stopping tablet replica
I20260806 22:50:08.049149 1943 raft_consensus.cc:2229] T 26cfc5f2554c401a9aa92e4ff5ddde90 P 89f290361bed4d6e83afd3250c4671c9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:50:08.049166 1941 raft_consensus.cc:2229] T 26cfc5f2554c401a9aa92e4ff5ddde90 P a52a8badd56f47fc9833529a2573dd5b [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:50:08.049252 1943 raft_consensus.cc:2258] T 26cfc5f2554c401a9aa92e4ff5ddde90 P 89f290361bed4d6e83afd3250c4671c9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:50:08.049400 1941 raft_consensus.cc:2258] T 26cfc5f2554c401a9aa92e4ff5ddde90 P a52a8badd56f47fc9833529a2573dd5b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:50:08.049425 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 1472
I20260806 22:50:08.050247 1941 ts_tablet_manager.cc:1905] T 26cfc5f2554c401a9aa92e4ff5ddde90 P a52a8badd56f47fc9833529a2573dd5b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:50:08.052240 1941 ts_tablet_manager.cc:1918] T 26cfc5f2554c401a9aa92e4ff5ddde90 P a52a8badd56f47fc9833529a2573dd5b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:50:08.052353 1941 log.cc:1196] T 26cfc5f2554c401a9aa92e4ff5ddde90 P a52a8badd56f47fc9833529a2573dd5b: Deleting WAL directory at /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056586788339-29547-0/minicluster-data/ts-2/wal/wals/26cfc5f2554c401a9aa92e4ff5ddde90
I20260806 22:50:08.052721 1941 ts_tablet_manager.cc:1939] T 26cfc5f2554c401a9aa92e4ff5ddde90 P a52a8badd56f47fc9833529a2573dd5b: Deleting consensus metadata
I20260806 22:50:08.053129 1391 catalog_manager.cc:4841] TS a52a8badd56f47fc9833529a2573dd5b (127.28.218.195:35743): tablet 26cfc5f2554c401a9aa92e4ff5ddde90 (table test-table [id=6af460fcab13486baaec9f457afb94ee]) successfully deleted
W20260806 22:50:08.057612 1392 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.28.218.193:42531 (error 108)
I20260806 22:50:08.057732 1942 tablet_replica.cc:330] T 26cfc5f2554c401a9aa92e4ff5ddde90 P 2d3b89d4880a4dfb875a5f8bab3d8063: stopping tablet replica
I20260806 22:50:08.057842 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 1616
I20260806 22:50:08.057883 1942 raft_consensus.cc:2229] T 26cfc5f2554c401a9aa92e4ff5ddde90 P 2d3b89d4880a4dfb875a5f8bab3d8063 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:50:08.058076 1942 raft_consensus.cc:2258] T 26cfc5f2554c401a9aa92e4ff5ddde90 P 2d3b89d4880a4dfb875a5f8bab3d8063 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:50:08.059197 1911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:08.058047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.193:42531 (local address 127.0.0.1:47378)
0806 22:50:08.059099 (+ 1052us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:08.059159 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.193:42531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1033}
W20260806 22:50:08.062955 1392 catalog_manager.cc:4584] TS 89f290361bed4d6e83afd3250c4671c9 (127.28.218.193:42531): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 26cfc5f2554c401a9aa92e4ff5ddde90: Network error: Client connection negotiation failed: client connection to 127.28.218.193:42531: connect: Connection refused (error 111)
I20260806 22:50:08.063024 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 1765
I20260806 22:50:08.065105 1911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:08.064985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.194:39307 (local address 127.0.0.1:43454)
0806 22:50:08.065051 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:08.065075 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.194:39307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260806 22:50:08.068388 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 1374
2026-08-06T22:50:08Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (1933 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-06T22:50:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:50:08Z Disabled control of system clock
I20260806 22:50:08.096465 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056586788339-29547-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056586788339-29547-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056586788339-29547-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.218.254:42807
--webserver_interface=127.28.218.254
--webserver_port=0
--builtin_ntp_servers=127.28.218.212:33093
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.218.254:42807
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260806 22:50:08.175163 1948 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:50:08.175349 1948 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:50:08.175370 1948 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:50:08.176432 1948 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:50:08.176468 1948 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:50:08.176481 1948 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:50:08.176494 1948 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:50:08.176506 1948 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:50:08.177750 1948 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:33093
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056586788339-29547-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056586788339-29547-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.218.254:42807
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.218.254:42807
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.218.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1948
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056586788339-29547-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:50:08.177928 1948 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:50:08.178143 1948 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:50:08.182652 1954 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:50:08.182744 1948 server_base.cc:1034] running on GCE node
W20260806 22:50:08.182747 1953 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:08.182646 1956 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:50:08.183104 1948 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:50:08.183418 1948 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:50:08.184583 1948 hybrid_clock.cc:648] HybridClock initialized: now 1786056608184564 us; error 36 us; skew 500 ppm
I20260806 22:50:08.185793 1948 webserver.cc:460] Webserver started at http://127.28.218.254:33509/ using document root <none> and password file <none>
I20260806 22:50:08.186028 1948 fs_manager.cc:359] Metadata directory not provided
I20260806 22:50:08.186095 1948 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:50:08.186241 1948 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:50:08.187288 1948 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056586788339-29547-0/minicluster-data/master-0/data/instance:
uuid: "fc4c076baf1544a699451272bd9943bb"
format_stamp: "Formatted at 2026-08-06 22:50:08 on dist-test-slave-1xrh"
server_key: "d99a00cc9683b2ea907b3d823eaf8cde"
server_key_iv: "d2d868d81cb687132f2dbe16e62e8211"
server_key_version: "encryptionkey@0"
I20260806 22:50:08.187635 1948 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056586788339-29547-0/minicluster-data/master-0/wal/instance:
uuid: "fc4c076baf1544a699451272bd9943bb"
format_stamp: "Formatted at 2026-08-06 22:50:08 on dist-test-slave-1xrh"
server_key: "d99a00cc9683b2ea907b3d823eaf8cde"
server_key_iv: "d2d868d81cb687132f2dbe16e62e8211"
server_key_version: "encryptionkey@0"
I20260806 22:50:08.188835 1948 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:50:08.189546 1964 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:08.189699 1948 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:50:08.189775 1948 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056586788339-29547-0/minicluster-data/master-0/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056586788339-29547-0/minicluster-data/master-0/wal
uuid: "fc4c076baf1544a699451272bd9943bb"
format_stamp: "Formatted at 2026-08-06 22:50:08 on dist-test-slave-1xrh"
server_key: "d99a00cc9683b2ea907b3d823eaf8cde"
server_key_iv: "d2d868d81cb687132f2dbe16e62e8211"
server_key_version: "encryptionkey@0"
I20260806 22:50:08.189826 1948 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056586788339-29547-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056586788339-29547-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056586788339-29547-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:50:08.192973 1948 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:50:08.193168 1948 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:50:08.193252 1948 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:50:08.196923 1948 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.254:42807
I20260806 22:50:08.196945 2018 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.254:42807 every 8 connection(s)
I20260806 22:50:08.197255 1948 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
I20260806 22:50:08.197811 2019 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:50:08.200315 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 1948
I20260806 22:50:08.200430 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056586788339-29547-0/minicluster-data/master-0/wal/instance
I20260806 22:50:08.200466 2019 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc4c076baf1544a699451272bd9943bb: Bootstrap starting.
I20260806 22:50:08.200611 29547 external_mini_cluster.cc:1428] Setting key f3b02ae6bca998c0ba5117a81485a6f4
I20260806 22:50:08.201179 2019 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fc4c076baf1544a699451272bd9943bb: Neither blocks nor log segments found. Creating new log.
I20260806 22:50:08.201495 2019 log.cc:824] T 00000000000000000000000000000000 P fc4c076baf1544a699451272bd9943bb: Log is configured to *not* fsync() on all Append() calls
I20260806 22:50:08.202255 2019 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc4c076baf1544a699451272bd9943bb: No bootstrap required, opened a new log
I20260806 22:50:08.203045 2023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:08.201068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51830 (local address 127.28.218.254:42807)
0806 22:50:08.201275 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:08.201283 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:08.201316 (+ 33us) server_negotiation.cc:407] Connection header received
0806 22:50:08.201766 (+ 450us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:08.201775 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:08.201822 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:08.201973 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:08.202633 (+ 660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:50:08.202637 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:50:08.202637 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:50:08.202641 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:50:08.202686 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:50:08.202701 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:50:08.202791 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:08.202818 (+ 27us) server_negotiation.cc:299] Negotiation successful
0806 22:50:08.202867 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":74,"threads_started":1}
I20260806 22:50:08.203995 2019 raft_consensus.cc:348] T 00000000000000000000000000000000 P fc4c076baf1544a699451272bd9943bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc4c076baf1544a699451272bd9943bb" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 42807 } }
I20260806 22:50:08.204113 2019 raft_consensus.cc:374] T 00000000000000000000000000000000 P fc4c076baf1544a699451272bd9943bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:50:08.204135 2019 raft_consensus.cc:729] T 00000000000000000000000000000000 P fc4c076baf1544a699451272bd9943bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc4c076baf1544a699451272bd9943bb, State: Initialized, Role: FOLLOWER
I20260806 22:50:08.204274 2019 consensus_queue.cc:260] T 00000000000000000000000000000000 P fc4c076baf1544a699451272bd9943bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc4c076baf1544a699451272bd9943bb" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 42807 } }
I20260806 22:50:08.204344 2019 raft_consensus.cc:388] T 00000000000000000000000000000000 P fc4c076baf1544a699451272bd9943bb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:50:08.204382 2019 raft_consensus.cc:482] T 00000000000000000000000000000000 P fc4c076baf1544a699451272bd9943bb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:50:08.204429 2019 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fc4c076baf1544a699451272bd9943bb [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:50:08.205199 2019 raft_consensus.cc:504] T 00000000000000000000000000000000 P fc4c076baf1544a699451272bd9943bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc4c076baf1544a699451272bd9943bb" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 42807 } }
I20260806 22:50:08.205300 2019 leader_election.cc:302] T 00000000000000000000000000000000 P fc4c076baf1544a699451272bd9943bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fc4c076baf1544a699451272bd9943bb; no voters:
I20260806 22:50:08.205497 2019 leader_election.cc:288] T 00000000000000000000000000000000 P fc4c076baf1544a699451272bd9943bb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:50:08.205554 2025 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fc4c076baf1544a699451272bd9943bb [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:50:08.205689 2025 raft_consensus.cc:686] T 00000000000000000000000000000000 P fc4c076baf1544a699451272bd9943bb [term 1 LEADER]: Becoming Leader. State: Replica: fc4c076baf1544a699451272bd9943bb, State: Running, Role: LEADER
I20260806 22:50:08.205735 2019 sys_catalog.cc:564] T 00000000000000000000000000000000 P fc4c076baf1544a699451272bd9943bb [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:50:08.205787 2025 consensus_queue.cc:237] T 00000000000000000000000000000000 P fc4c076baf1544a699451272bd9943bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc4c076baf1544a699451272bd9943bb" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 42807 } }
I20260806 22:50:08.206144 2026 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc4c076baf1544a699451272bd9943bb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fc4c076baf1544a699451272bd9943bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc4c076baf1544a699451272bd9943bb" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 42807 } } }
I20260806 22:50:08.206148 2027 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc4c076baf1544a699451272bd9943bb [sys.catalog]: SysCatalogTable state changed. Reason: New leader fc4c076baf1544a699451272bd9943bb. Latest consensus state: current_term: 1 leader_uuid: "fc4c076baf1544a699451272bd9943bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc4c076baf1544a699451272bd9943bb" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 42807 } } }
I20260806 22:50:08.206275 2026 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc4c076baf1544a699451272bd9943bb [sys.catalog]: This master's current role is: LEADER
I20260806 22:50:08.206283 2027 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc4c076baf1544a699451272bd9943bb [sys.catalog]: This master's current role is: LEADER
I20260806 22:50:08.206501 2033 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:50:08.206984 2033 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:50:08.208412 2033 catalog_manager.cc:1347] Generated new cluster ID: 6a019f6c10e34441bf4d1a16d3f63c45
I20260806 22:50:08.208465 2033 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:50:08.217792 2033 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:50:08.217944 2033 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:50:08.221410 2033 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fc4c076baf1544a699451272bd9943bb: Generated new TSK 0
I20260806 22:50:08.221583 2033 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:50:08.228991 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056586788339-29547-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056586788339-29547-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056586788339-29547-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056586788339-29547-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.218.193:0
--local_ip_for_outbound_sockets=127.28.218.193
--webserver_interface=127.28.218.193
--webserver_port=0
--tserver_master_addrs=127.28.218.254:42807
--builtin_ntp_servers=127.28.218.212:33093
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260806 22:50:08.308820 2045 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:50:08.309027 2045 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:50:08.309065 2045 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:50:08.310675 2045 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:50:08.310729 2045 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:50:08.310765 2045 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.218.193
I20260806 22:50:08.312799 2045 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:33093
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056586788339-29547-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056586788339-29547-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.218.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056586788339-29547-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.218.193
--webserver_port=0
--tserver_master_addrs=127.28.218.254:42807
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2045
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.218.193
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056586788339-29547-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:50:08.313055 2045 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:50:08.313392 2045 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:50:08.319850 2053 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:50:08.320089 2045 server_base.cc:1034] running on GCE node
W20260806 22:50:08.319824 2051 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:08.320753 2050 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:50:08.321305 2045 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:50:08.321578 2045 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:50:08.322726 2045 hybrid_clock.cc:648] HybridClock initialized: now 1786056608322716 us; error 25 us; skew 500 ppm
I20260806 22:50:08.324195 2045 webserver.cc:460] Webserver started at http://127.28.218.193:34821/ using document root <none> and password file <none>
I20260806 22:50:08.324498 2045 fs_manager.cc:359] Metadata directory not provided
I20260806 22:50:08.324568 2045 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:50:08.324712 2045 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:50:08.325780 2045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056586788339-29547-0/minicluster-data/ts-0/data/instance:
uuid: "657f8e9000d2486586248a2d32e76c3d"
format_stamp: "Formatted at 2026-08-06 22:50:08 on dist-test-slave-1xrh"
server_key: "103ebedd7e2a3114d5877890cb4de6dd"
server_key_iv: "3343e3d80536b3e0ad15dc6620f33678"
server_key_version: "encryptionkey@0"
I20260806 22:50:08.326148 2045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056586788339-29547-0/minicluster-data/ts-0/wal/instance:
uuid: "657f8e9000d2486586248a2d32e76c3d"
format_stamp: "Formatted at 2026-08-06 22:50:08 on dist-test-slave-1xrh"
server_key: "103ebedd7e2a3114d5877890cb4de6dd"
server_key_iv: "3343e3d80536b3e0ad15dc6620f33678"
server_key_version: "encryptionkey@0"
I20260806 22:50:08.327590 2045 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 22:50:08.328470 2059 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:08.328641 2045 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:50:08.328727 2045 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056586788339-29547-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056586788339-29547-0/minicluster-data/ts-0/wal
uuid: "657f8e9000d2486586248a2d32e76c3d"
format_stamp: "Formatted at 2026-08-06 22:50:08 on dist-test-slave-1xrh"
server_key: "103ebedd7e2a3114d5877890cb4de6dd"
server_key_iv: "3343e3d80536b3e0ad15dc6620f33678"
server_key_version: "encryptionkey@0"
I20260806 22:50:08.328809 2045 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056586788339-29547-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056586788339-29547-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056586788339-29547-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:50:08.333716 2045 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:50:08.334021 2045 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:50:08.334164 2045 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:50:08.334434 2045 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:50:08.334785 2045 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:50:08.334828 2045 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:08.334867 2045 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:50:08.334893 2045 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:08.342005 2045 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.193:46641
I20260806 22:50:08.342062 2176 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.193:46641 every 8 connection(s)
I20260806 22:50:08.342338 2045 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056586788339-29547-0/minicluster-data/ts-0/data/info.pb
I20260806 22:50:08.344563 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 2045
I20260806 22:50:08.344648 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056586788339-29547-0/minicluster-data/ts-0/wal/instance
I20260806 22:50:08.344830 29547 external_mini_cluster.cc:1428] Setting key 3a1494f754001b3effad52bae167ccf7
I20260806 22:50:08.345280 2023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:08.343068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.193:34259 (local address 127.28.218.254:42807)
0806 22:50:08.343111 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:08.343114 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:08.343381 (+ 267us) server_negotiation.cc:407] Connection header received
0806 22:50:08.343829 (+ 448us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:08.343830 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:08.343836 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:08.343887 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:08.345029 (+ 1142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:50:08.345031 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:50:08.345031 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:50:08.345036 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:50:08.345051 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:50:08.345055 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:50:08.345105 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:08.345217 (+ 112us) server_negotiation.cc:299] Negotiation successful
0806 22:50:08.345228 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 22:50:08.345504 2179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:08.343049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:42807 (local address 127.28.218.193:34259)
0806 22:50:08.343285 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:08.343315 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:08.343742 (+ 427us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:08.343936 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:08.343939 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:08.343947 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:08.344874 (+ 927us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:50:08.344875 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:08.344912 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:50:08.344945 (+ 33us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:50:08.345131 (+ 186us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:50:08.345139 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:50:08.345143 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:50:08.345222 (+ 79us) client_negotiation.cc:240] Negotiation successful
0806 22:50:08.345282 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":156,"threads_started":1}
I20260806 22:50:08.345768 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056586788339-29547-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056586788339-29547-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056586788339-29547-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056586788339-29547-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.218.194:0
--local_ip_for_outbound_sockets=127.28.218.194
--webserver_interface=127.28.218.194
--webserver_port=0
--tserver_master_addrs=127.28.218.254:42807
--builtin_ntp_servers=127.28.218.212:33093
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260806 22:50:08.347492 2177 heartbeater.cc:347] Connected to a master server at 127.28.218.254:42807
I20260806 22:50:08.347592 2177 heartbeater.cc:464] Registering TS with master...
I20260806 22:50:08.347741 2177 heartbeater.cc:511] Master 127.28.218.254:42807 requested a full tablet report, sending...
I20260806 22:50:08.348237 1982 ts_manager.cc:194] Registered new tserver with Master: 657f8e9000d2486586248a2d32e76c3d (127.28.218.193:46641)
W20260806 22:50:08.427609 2181 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:50:08.427809 2181 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:50:08.427830 2181 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:50:08.429023 2181 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:50:08.429061 2181 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:50:08.429109 2181 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.218.194
I20260806 22:50:08.430409 2181 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:33093
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056586788339-29547-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056586788339-29547-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.218.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056586788339-29547-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.218.194
--webserver_port=0
--tserver_master_addrs=127.28.218.254:42807
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2181
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.218.194
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056586788339-29547-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:50:08.430644 2181 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:50:08.430893 2181 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:50:08.435608 2190 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:08.435663 2188 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:08.435665 2187 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:50:08.435931 2181 server_base.cc:1034] running on GCE node
I20260806 22:50:08.436151 2181 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:50:08.436373 2181 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:50:08.437525 2181 hybrid_clock.cc:648] HybridClock initialized: now 1786056608437516 us; error 33 us; skew 500 ppm
I20260806 22:50:08.438673 2181 webserver.cc:460] Webserver started at http://127.28.218.194:45893/ using document root <none> and password file <none>
I20260806 22:50:08.438897 2181 fs_manager.cc:359] Metadata directory not provided
I20260806 22:50:08.438941 2181 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:50:08.439105 2181 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:50:08.440109 2181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056586788339-29547-0/minicluster-data/ts-1/data/instance:
uuid: "40923b8b73cf438e98102a8d3b558db2"
format_stamp: "Formatted at 2026-08-06 22:50:08 on dist-test-slave-1xrh"
server_key: "1af4f5289ad1319f7e703f9e00934edc"
server_key_iv: "246a5442218953af420d7f16c90fc6e5"
server_key_version: "encryptionkey@0"
I20260806 22:50:08.440450 2181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056586788339-29547-0/minicluster-data/ts-1/wal/instance:
uuid: "40923b8b73cf438e98102a8d3b558db2"
format_stamp: "Formatted at 2026-08-06 22:50:08 on dist-test-slave-1xrh"
server_key: "1af4f5289ad1319f7e703f9e00934edc"
server_key_iv: "246a5442218953af420d7f16c90fc6e5"
server_key_version: "encryptionkey@0"
I20260806 22:50:08.441694 2181 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 22:50:08.442330 2196 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:08.442479 2181 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:50:08.442557 2181 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056586788339-29547-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056586788339-29547-0/minicluster-data/ts-1/wal
uuid: "40923b8b73cf438e98102a8d3b558db2"
format_stamp: "Formatted at 2026-08-06 22:50:08 on dist-test-slave-1xrh"
server_key: "1af4f5289ad1319f7e703f9e00934edc"
server_key_iv: "246a5442218953af420d7f16c90fc6e5"
server_key_version: "encryptionkey@0"
I20260806 22:50:08.442608 2181 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056586788339-29547-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056586788339-29547-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056586788339-29547-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:50:08.445876 2181 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:50:08.446080 2181 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:50:08.446174 2181 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:50:08.446317 2181 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:50:08.446548 2181 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:50:08.446578 2181 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:08.446596 2181 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:50:08.446615 2181 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:08.452009 2181 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.194:40871
I20260806 22:50:08.452082 2313 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.194:40871 every 8 connection(s)
I20260806 22:50:08.452346 2181 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056586788339-29547-0/minicluster-data/ts-1/data/info.pb
I20260806 22:50:08.455022 2023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:08.452984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.194:47777 (local address 127.28.218.254:42807)
0806 22:50:08.453038 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:08.453040 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:08.453241 (+ 201us) server_negotiation.cc:407] Connection header received
0806 22:50:08.453676 (+ 435us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:08.453677 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:08.453681 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:08.453724 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:08.454860 (+ 1136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:50:08.454861 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:50:08.454861 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:50:08.454864 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:50:08.454873 (+ 9us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:50:08.454876 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:50:08.454913 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:08.454958 (+ 45us) server_negotiation.cc:299] Negotiation successful
0806 22:50:08.454969 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260806 22:50:08.455251 2316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:08.452935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:42807 (local address 127.28.218.194:47777)
0806 22:50:08.453106 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:08.453142 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:08.453584 (+ 442us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:08.453789 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:08.453793 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:08.453800 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:08.454745 (+ 945us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:50:08.454745 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:08.454771 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:50:08.454796 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:50:08.454913 (+ 117us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:50:08.454915 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:50:08.454917 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 22:50:08.454965 (+ 48us) client_negotiation.cc:240] Negotiation successful
0806 22:50:08.455007 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":90,"threads_started":1}
I20260806 22:50:08.455744 2314 heartbeater.cc:347] Connected to a master server at 127.28.218.254:42807
I20260806 22:50:08.455833 2314 heartbeater.cc:464] Registering TS with master...
I20260806 22:50:08.455968 2314 heartbeater.cc:511] Master 127.28.218.254:42807 requested a full tablet report, sending...
I20260806 22:50:08.456322 1982 ts_manager.cc:194] Registered new tserver with Master: 40923b8b73cf438e98102a8d3b558db2 (127.28.218.194:40871)
I20260806 22:50:08.461006 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 2181
I20260806 22:50:08.461087 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056586788339-29547-0/minicluster-data/ts-1/wal/instance
I20260806 22:50:08.461257 29547 external_mini_cluster.cc:1428] Setting key 30dedf02b0fb1bb5545a15b42ab964f6
I20260806 22:50:08.462119 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056586788339-29547-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056586788339-29547-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056586788339-29547-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056586788339-29547-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.218.195:0
--local_ip_for_outbound_sockets=127.28.218.195
--webserver_interface=127.28.218.195
--webserver_port=0
--tserver_master_addrs=127.28.218.254:42807
--builtin_ntp_servers=127.28.218.212:33093
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260806 22:50:08.541575 2317 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:50:08.541752 2317 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:50:08.541774 2317 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:50:08.542807 2317 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:50:08.542838 2317 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:50:08.542862 2317 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.218.195
I20260806 22:50:08.544189 2317 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:33093
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056586788339-29547-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056586788339-29547-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.218.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056586788339-29547-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.218.195
--webserver_port=0
--tserver_master_addrs=127.28.218.254:42807
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2317
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.218.195
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056586788339-29547-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:50:08.544360 2317 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:50:08.544577 2317 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:50:08.549275 2325 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:08.549324 2323 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:50:08.549456 2317 server_base.cc:1034] running on GCE node
W20260806 22:50:08.549512 2322 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:50:08.549752 2317 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:50:08.549976 2317 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:50:08.551167 2317 hybrid_clock.cc:648] HybridClock initialized: now 1786056608551157 us; error 35 us; skew 500 ppm
I20260806 22:50:08.552348 2317 webserver.cc:460] Webserver started at http://127.28.218.195:40089/ using document root <none> and password file <none>
I20260806 22:50:08.552564 2317 fs_manager.cc:359] Metadata directory not provided
I20260806 22:50:08.552627 2317 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:50:08.552767 2317 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:50:08.553730 2317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056586788339-29547-0/minicluster-data/ts-2/data/instance:
uuid: "6bfb041fcf764f73b5227cfee983bb3f"
format_stamp: "Formatted at 2026-08-06 22:50:08 on dist-test-slave-1xrh"
server_key: "344146fc87c573ee950b6616675618bf"
server_key_iv: "e45aaf6c3809b759ee1b8743fe8bd19a"
server_key_version: "encryptionkey@0"
I20260806 22:50:08.554073 2317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056586788339-29547-0/minicluster-data/ts-2/wal/instance:
uuid: "6bfb041fcf764f73b5227cfee983bb3f"
format_stamp: "Formatted at 2026-08-06 22:50:08 on dist-test-slave-1xrh"
server_key: "344146fc87c573ee950b6616675618bf"
server_key_iv: "e45aaf6c3809b759ee1b8743fe8bd19a"
server_key_version: "encryptionkey@0"
I20260806 22:50:08.555323 2317 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:50:08.555981 2334 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:08.556128 2317 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:50:08.556227 2317 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056586788339-29547-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056586788339-29547-0/minicluster-data/ts-2/wal
uuid: "6bfb041fcf764f73b5227cfee983bb3f"
format_stamp: "Formatted at 2026-08-06 22:50:08 on dist-test-slave-1xrh"
server_key: "344146fc87c573ee950b6616675618bf"
server_key_iv: "e45aaf6c3809b759ee1b8743fe8bd19a"
server_key_version: "encryptionkey@0"
I20260806 22:50:08.556296 2317 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056586788339-29547-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056586788339-29547-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056586788339-29547-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:50:08.559485 2317 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:50:08.559731 2317 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:50:08.559862 2317 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:50:08.560062 2317 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:50:08.560343 2317 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:50:08.560392 2317 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:08.560431 2317 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:50:08.560458 2317 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:08.566184 2317 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.195:34537
I20260806 22:50:08.566231 2455 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.195:34537 every 8 connection(s)
I20260806 22:50:08.566510 2317 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056586788339-29547-0/minicluster-data/ts-2/data/info.pb
I20260806 22:50:08.566969 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 2317
I20260806 22:50:08.567085 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056586788339-29547-0/minicluster-data/ts-2/wal/instance
I20260806 22:50:08.567291 29547 external_mini_cluster.cc:1428] Setting key 1e6b6cd6adef59c4bf214c3c4d7c3295
I20260806 22:50:08.569514 2023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:08.567183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.195:43559 (local address 127.28.218.254:42807)
0806 22:50:08.567237 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:08.567240 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:08.567607 (+ 367us) server_negotiation.cc:407] Connection header received
0806 22:50:08.568160 (+ 553us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:08.568161 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:08.568167 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:08.568214 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:08.569259 (+ 1045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:50:08.569260 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:50:08.569261 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:50:08.569266 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:50:08.569281 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:50:08.569285 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:50:08.569354 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:08.569426 (+ 72us) server_negotiation.cc:299] Negotiation successful
0806 22:50:08.569440 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260806 22:50:08.569686 2458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:08.567165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:42807 (local address 127.28.218.195:43559)
0806 22:50:08.567352 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:08.567395 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:08.568062 (+ 667us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:08.568275 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:08.568278 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:08.568287 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:08.569121 (+ 834us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:50:08.569122 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:08.569148 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:50:08.569173 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:50:08.569354 (+ 181us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:50:08.569358 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:50:08.569361 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:50:08.569435 (+ 74us) client_negotiation.cc:240] Negotiation successful
0806 22:50:08.569488 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":74,"threads_started":1}
I20260806 22:50:08.570083 2456 heartbeater.cc:347] Connected to a master server at 127.28.218.254:42807
I20260806 22:50:08.570151 2456 heartbeater.cc:464] Registering TS with master...
I20260806 22:50:08.570295 2456 heartbeater.cc:511] Master 127.28.218.254:42807 requested a full tablet report, sending...
I20260806 22:50:08.570593 1982 ts_manager.cc:194] Registered new tserver with Master: 6bfb041fcf764f73b5227cfee983bb3f (127.28.218.195:34537)
I20260806 22:50:08.579038 29547 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:50:08.581502 2023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:08.580395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51844 (local address 127.28.218.254:42807)
0806 22:50:08.580468 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:08.580470 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:08.580581 (+ 111us) server_negotiation.cc:407] Connection header received
0806 22:50:08.580710 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:08.580711 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:08.580717 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:08.580769 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:08.581284 (+ 515us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:50:08.581286 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:50:08.581286 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:50:08.581290 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:50:08.581303 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:50:08.581307 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:50:08.581352 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:08.581415 (+ 63us) server_negotiation.cc:299] Negotiation successful
0806 22:50:08.581430 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 22:50:08.582897 1982 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51844:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:50:08.584533 1982 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:50:08.588081 2468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:08.586545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.193:46641 (local address 127.0.0.1:46484)
0806 22:50:08.586850 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:08.586861 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:08.586909 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:08.587113 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:08.587139 (+ 26us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:08.587146 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:08.587649 (+ 503us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:50:08.587650 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:08.587664 (+ 14us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:50:08.587687 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:50:08.587889 (+ 202us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:50:08.587896 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:50:08.587899 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:50:08.587988 (+ 89us) client_negotiation.cc:240] Negotiation successful
0806 22:50:08.588001 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"mutex_wait_us":4,"thread_start_us":70,"threads_started":1}
I20260806 22:50:08.588136 2469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:08.586731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46484 (local address 127.28.218.193:46641)
0806 22:50:08.586940 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:08.586945 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:08.586956 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:50:08.586980 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:08.586982 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:08.586992 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:08.587347 (+ 355us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:08.587753 (+ 406us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:50:08.587760 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:50:08.587760 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:50:08.587764 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:50:08.587792 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:50:08.587811 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:50:08.587891 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:08.587996 (+ 105us) server_negotiation.cc:299] Negotiation successful
0806 22:50:08.588014 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":95,"threads_started":1}
I20260806 22:50:08.588370 2466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:08.586544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.194:40871 (local address 127.0.0.1:51564)
0806 22:50:08.586703 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:08.586739 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:08.586806 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:08.587439 (+ 633us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:08.587440 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:08.587442 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:08.587936 (+ 494us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:50:08.587936 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:08.587941 (+ 5us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:50:08.587962 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:50:08.588252 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:50:08.588253 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:50:08.588254 (+ 1us) client_negotiation.cc:769] Sending connection context
0806 22:50:08.588292 (+ 38us) client_negotiation.cc:240] Negotiation successful
0806 22:50:08.588305 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":53,"threads_started":1}
I20260806 22:50:08.588613 2465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:08.586722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.195:34537 (local address 127.0.0.1:51470)
0806 22:50:08.586929 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:08.586940 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:08.586992 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:08.587206 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:08.587207 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:08.587209 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:08.588238 (+ 1029us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:50:08.588239 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:08.588247 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:50:08.588262 (+ 15us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:50:08.588491 (+ 229us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:50:08.588491 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:50:08.588492 (+ 1us) client_negotiation.cc:769] Sending connection context
0806 22:50:08.588550 (+ 58us) client_negotiation.cc:240] Negotiation successful
0806 22:50:08.588561 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":77,"threads_started":1}
I20260806 22:50:08.588617 2467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:08.586636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51564 (local address 127.28.218.194:40871)
0806 22:50:08.587104 (+ 468us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:08.587107 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:08.587116 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:50:08.587195 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:08.587198 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:08.587207 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:08.587282 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:08.588053 (+ 771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:50:08.588059 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:50:08.588060 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:50:08.588064 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:50:08.588104 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:50:08.588130 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:50:08.588186 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:08.588512 (+ 326us) server_negotiation.cc:299] Negotiation successful
0806 22:50:08.588530 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":436,"thread_start_us":97,"threads_started":1}
I20260806 22:50:08.588807 2470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:08.586904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51470 (local address 127.28.218.195:34537)
0806 22:50:08.587042 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:08.587046 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:08.587055 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:50:08.587077 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:08.587079 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:08.587088 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:08.587203 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:08.588347 (+ 1144us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:50:08.588352 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:50:08.588353 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:50:08.588356 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:50:08.588387 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:50:08.588408 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:50:08.588463 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:08.588708 (+ 245us) server_negotiation.cc:299] Negotiation successful
0806 22:50:08.588727 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":71,"threads_started":1}
I20260806 22:50:08.588924 2110 tablet_service.cc:1467] Processing CreateTablet for tablet 90495bf3f6144afb9590ae87943161ac (DEFAULT_TABLE table=test-table [id=73607dda279549f9bc7847bd14cf4ba7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:50:08.589207 2246 tablet_service.cc:1467] Processing CreateTablet for tablet 90495bf3f6144afb9590ae87943161ac (DEFAULT_TABLE table=test-table [id=73607dda279549f9bc7847bd14cf4ba7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:50:08.589300 2110 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 90495bf3f6144afb9590ae87943161ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:50:08.589512 2246 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 90495bf3f6144afb9590ae87943161ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:50:08.592069 2471 tablet_bootstrap.cc:492] T 90495bf3f6144afb9590ae87943161ac P 657f8e9000d2486586248a2d32e76c3d: Bootstrap starting.
I20260806 22:50:08.592489 2387 tablet_service.cc:1467] Processing CreateTablet for tablet 90495bf3f6144afb9590ae87943161ac (DEFAULT_TABLE table=test-table [id=73607dda279549f9bc7847bd14cf4ba7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:50:08.592767 2387 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 90495bf3f6144afb9590ae87943161ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:50:08.594965 2473 tablet_bootstrap.cc:492] T 90495bf3f6144afb9590ae87943161ac P 6bfb041fcf764f73b5227cfee983bb3f: Bootstrap starting.
I20260806 22:50:08.595609 2471 tablet_bootstrap.cc:654] T 90495bf3f6144afb9590ae87943161ac P 657f8e9000d2486586248a2d32e76c3d: Neither blocks nor log segments found. Creating new log.
I20260806 22:50:08.595609 2473 tablet_bootstrap.cc:654] T 90495bf3f6144afb9590ae87943161ac P 6bfb041fcf764f73b5227cfee983bb3f: Neither blocks nor log segments found. Creating new log.
I20260806 22:50:08.595943 2473 log.cc:824] T 90495bf3f6144afb9590ae87943161ac P 6bfb041fcf764f73b5227cfee983bb3f: Log is configured to *not* fsync() on all Append() calls
I20260806 22:50:08.595944 2471 log.cc:824] T 90495bf3f6144afb9590ae87943161ac P 657f8e9000d2486586248a2d32e76c3d: Log is configured to *not* fsync() on all Append() calls
I20260806 22:50:08.596267 2472 tablet_bootstrap.cc:492] T 90495bf3f6144afb9590ae87943161ac P 40923b8b73cf438e98102a8d3b558db2: Bootstrap starting.
I20260806 22:50:08.596773 2471 tablet_bootstrap.cc:492] T 90495bf3f6144afb9590ae87943161ac P 657f8e9000d2486586248a2d32e76c3d: No bootstrap required, opened a new log
I20260806 22:50:08.596800 2473 tablet_bootstrap.cc:492] T 90495bf3f6144afb9590ae87943161ac P 6bfb041fcf764f73b5227cfee983bb3f: No bootstrap required, opened a new log
I20260806 22:50:08.596850 2471 ts_tablet_manager.cc:1397] T 90495bf3f6144afb9590ae87943161ac P 657f8e9000d2486586248a2d32e76c3d: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260806 22:50:08.596869 2473 ts_tablet_manager.cc:1397] T 90495bf3f6144afb9590ae87943161ac P 6bfb041fcf764f73b5227cfee983bb3f: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260806 22:50:08.596871 2472 tablet_bootstrap.cc:654] T 90495bf3f6144afb9590ae87943161ac P 40923b8b73cf438e98102a8d3b558db2: Neither blocks nor log segments found. Creating new log.
I20260806 22:50:08.597184 2472 log.cc:824] T 90495bf3f6144afb9590ae87943161ac P 40923b8b73cf438e98102a8d3b558db2: Log is configured to *not* fsync() on all Append() calls
I20260806 22:50:08.597936 2472 tablet_bootstrap.cc:492] T 90495bf3f6144afb9590ae87943161ac P 40923b8b73cf438e98102a8d3b558db2: No bootstrap required, opened a new log
I20260806 22:50:08.598011 2472 ts_tablet_manager.cc:1397] T 90495bf3f6144afb9590ae87943161ac P 40923b8b73cf438e98102a8d3b558db2: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:50:08.599016 2473 raft_consensus.cc:348] T 90495bf3f6144afb9590ae87943161ac P 6bfb041fcf764f73b5227cfee983bb3f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "657f8e9000d2486586248a2d32e76c3d" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 46641 } } peers { permanent_uuid: "40923b8b73cf438e98102a8d3b558db2" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 40871 } } peers { permanent_uuid: "6bfb041fcf764f73b5227cfee983bb3f" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 34537 } }
I20260806 22:50:08.599180 2473 raft_consensus.cc:374] T 90495bf3f6144afb9590ae87943161ac P 6bfb041fcf764f73b5227cfee983bb3f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:50:08.599226 2473 raft_consensus.cc:729] T 90495bf3f6144afb9590ae87943161ac P 6bfb041fcf764f73b5227cfee983bb3f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bfb041fcf764f73b5227cfee983bb3f, State: Initialized, Role: FOLLOWER
I20260806 22:50:08.599339 2473 consensus_queue.cc:260] T 90495bf3f6144afb9590ae87943161ac P 6bfb041fcf764f73b5227cfee983bb3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "657f8e9000d2486586248a2d32e76c3d" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 46641 } } peers { permanent_uuid: "40923b8b73cf438e98102a8d3b558db2" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 40871 } } peers { permanent_uuid: "6bfb041fcf764f73b5227cfee983bb3f" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 34537 } }
I20260806 22:50:08.599434 2471 raft_consensus.cc:348] T 90495bf3f6144afb9590ae87943161ac P 657f8e9000d2486586248a2d32e76c3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "657f8e9000d2486586248a2d32e76c3d" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 46641 } } peers { permanent_uuid: "40923b8b73cf438e98102a8d3b558db2" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 40871 } } peers { permanent_uuid: "6bfb041fcf764f73b5227cfee983bb3f" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 34537 } }
I20260806 22:50:08.599582 2473 ts_tablet_manager.cc:1428] T 90495bf3f6144afb9590ae87943161ac P 6bfb041fcf764f73b5227cfee983bb3f: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260806 22:50:08.599606 2471 raft_consensus.cc:374] T 90495bf3f6144afb9590ae87943161ac P 657f8e9000d2486586248a2d32e76c3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:50:08.599642 2471 raft_consensus.cc:729] T 90495bf3f6144afb9590ae87943161ac P 657f8e9000d2486586248a2d32e76c3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 657f8e9000d2486586248a2d32e76c3d, State: Initialized, Role: FOLLOWER
I20260806 22:50:08.599711 2456 heartbeater.cc:503] Master 127.28.218.254:42807 was elected leader, sending a full tablet report...
I20260806 22:50:08.599758 2471 consensus_queue.cc:260] T 90495bf3f6144afb9590ae87943161ac P 657f8e9000d2486586248a2d32e76c3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "657f8e9000d2486586248a2d32e76c3d" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 46641 } } peers { permanent_uuid: "40923b8b73cf438e98102a8d3b558db2" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 40871 } } peers { permanent_uuid: "6bfb041fcf764f73b5227cfee983bb3f" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 34537 } }
I20260806 22:50:08.600018 2471 ts_tablet_manager.cc:1428] T 90495bf3f6144afb9590ae87943161ac P 657f8e9000d2486586248a2d32e76c3d: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260806 22:50:08.600040 2177 heartbeater.cc:503] Master 127.28.218.254:42807 was elected leader, sending a full tablet report...
I20260806 22:50:08.601197 2472 raft_consensus.cc:348] T 90495bf3f6144afb9590ae87943161ac P 40923b8b73cf438e98102a8d3b558db2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "657f8e9000d2486586248a2d32e76c3d" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 46641 } } peers { permanent_uuid: "40923b8b73cf438e98102a8d3b558db2" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 40871 } } peers { permanent_uuid: "6bfb041fcf764f73b5227cfee983bb3f" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 34537 } }
I20260806 22:50:08.601307 2472 raft_consensus.cc:374] T 90495bf3f6144afb9590ae87943161ac P 40923b8b73cf438e98102a8d3b558db2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:50:08.601327 2472 raft_consensus.cc:729] T 90495bf3f6144afb9590ae87943161ac P 40923b8b73cf438e98102a8d3b558db2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40923b8b73cf438e98102a8d3b558db2, State: Initialized, Role: FOLLOWER
I20260806 22:50:08.601392 2472 consensus_queue.cc:260] T 90495bf3f6144afb9590ae87943161ac P 40923b8b73cf438e98102a8d3b558db2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "657f8e9000d2486586248a2d32e76c3d" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 46641 } } peers { permanent_uuid: "40923b8b73cf438e98102a8d3b558db2" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 40871 } } peers { permanent_uuid: "6bfb041fcf764f73b5227cfee983bb3f" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 34537 } }
I20260806 22:50:08.601557 2472 ts_tablet_manager.cc:1428] T 90495bf3f6144afb9590ae87943161ac P 40923b8b73cf438e98102a8d3b558db2: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260806 22:50:08.601614 2314 heartbeater.cc:503] Master 127.28.218.254:42807 was elected leader, sending a full tablet report...
I20260806 22:50:08.700932 2479 raft_consensus.cc:482] T 90495bf3f6144afb9590ae87943161ac P 6bfb041fcf764f73b5227cfee983bb3f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:50:08.701062 2479 raft_consensus.cc:504] T 90495bf3f6144afb9590ae87943161ac P 6bfb041fcf764f73b5227cfee983bb3f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "657f8e9000d2486586248a2d32e76c3d" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 46641 } } peers { permanent_uuid: "40923b8b73cf438e98102a8d3b558db2" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 40871 } } peers { permanent_uuid: "6bfb041fcf764f73b5227cfee983bb3f" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 34537 } }
I20260806 22:50:08.701510 2479 leader_election.cc:288] T 90495bf3f6144afb9590ae87943161ac P 6bfb041fcf764f73b5227cfee983bb3f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 657f8e9000d2486586248a2d32e76c3d (127.28.218.193:46641), 40923b8b73cf438e98102a8d3b558db2 (127.28.218.194:40871)
I20260806 22:50:08.702759 2458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:08.701679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.193:46641 (local address 127.28.218.195:54045)
0806 22:50:08.701728 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:08.701742 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:08.701780 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:08.701930 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:08.701931 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:08.701933 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:08.702377 (+ 444us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:50:08.702377 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:08.702384 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:50:08.702395 (+ 11us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:50:08.702602 (+ 207us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:50:08.702603 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:50:08.702604 (+ 1us) client_negotiation.cc:769] Sending connection context
0806 22:50:08.702652 (+ 48us) client_negotiation.cc:240] Negotiation successful
0806 22:50:08.702666 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260806 22:50:08.702760 2469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:08.701766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.195:54045 (local address 127.28.218.193:46641)
0806 22:50:08.701813 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:08.701815 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:08.701826 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:50:08.701854 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:08.701855 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:08.701861 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:08.701922 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:08.702500 (+ 578us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:50:08.702502 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:50:08.702502 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:50:08.702507 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:50:08.702526 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:50:08.702530 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:50:08.702583 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:08.702654 (+ 71us) server_negotiation.cc:299] Negotiation successful
0806 22:50:08.702669 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
W20260806 22:50:08.702864 2315 tablet.cc:2369] T 90495bf3f6144afb9590ae87943161ac P 40923b8b73cf438e98102a8d3b558db2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:50:08.702994 2482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:08.701680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.194:40871 (local address 127.28.218.195:59591)
0806 22:50:08.701876 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:08.701885 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:08.701928 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:08.702074 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:08.702075 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:08.702078 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:08.702543 (+ 465us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:50:08.702543 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:08.702547 (+ 4us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:50:08.702555 (+ 8us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:50:08.702735 (+ 180us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:50:08.702736 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:50:08.702736 (+ 0us) client_negotiation.cc:769] Sending connection context
0806 22:50:08.702892 (+ 156us) client_negotiation.cc:240] Negotiation successful
0806 22:50:08.702907 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"mutex_wait_us":40,"thread_start_us":116,"threads_started":1}
I20260806 22:50:08.703208 2131 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90495bf3f6144afb9590ae87943161ac" candidate_uuid: "6bfb041fcf764f73b5227cfee983bb3f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657f8e9000d2486586248a2d32e76c3d" is_pre_election: true
I20260806 22:50:08.703306 2467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:08.701798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.195:59591 (local address 127.28.218.194:40871)
0806 22:50:08.701845 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:08.701846 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:08.701939 (+ 93us) server_negotiation.cc:407] Connection header received
0806 22:50:08.701993 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:08.701994 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:08.701999 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:08.702057 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:08.702644 (+ 587us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:50:08.702645 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:50:08.702646 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:50:08.702650 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:50:08.702666 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:50:08.702671 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:50:08.702740 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:08.702821 (+ 81us) server_negotiation.cc:299] Negotiation successful
0806 22:50:08.702836 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 22:50:08.703367 2131 raft_consensus.cc:2454] T 90495bf3f6144afb9590ae87943161ac P 657f8e9000d2486586248a2d32e76c3d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6bfb041fcf764f73b5227cfee983bb3f in term 0.
I20260806 22:50:08.703572 2268 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90495bf3f6144afb9590ae87943161ac" candidate_uuid: "6bfb041fcf764f73b5227cfee983bb3f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40923b8b73cf438e98102a8d3b558db2" is_pre_election: true
I20260806 22:50:08.703620 2336 leader_election.cc:302] T 90495bf3f6144afb9590ae87943161ac P 6bfb041fcf764f73b5227cfee983bb3f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 657f8e9000d2486586248a2d32e76c3d, 6bfb041fcf764f73b5227cfee983bb3f; no voters:
I20260806 22:50:08.703711 2268 raft_consensus.cc:2454] T 90495bf3f6144afb9590ae87943161ac P 40923b8b73cf438e98102a8d3b558db2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6bfb041fcf764f73b5227cfee983bb3f in term 0.
I20260806 22:50:08.703735 2479 raft_consensus.cc:2781] T 90495bf3f6144afb9590ae87943161ac P 6bfb041fcf764f73b5227cfee983bb3f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:50:08.703790 2479 raft_consensus.cc:482] T 90495bf3f6144afb9590ae87943161ac P 6bfb041fcf764f73b5227cfee983bb3f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:50:08.703837 2479 raft_consensus.cc:3037] T 90495bf3f6144afb9590ae87943161ac P 6bfb041fcf764f73b5227cfee983bb3f [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:50:08.704548 2479 raft_consensus.cc:504] T 90495bf3f6144afb9590ae87943161ac P 6bfb041fcf764f73b5227cfee983bb3f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "657f8e9000d2486586248a2d32e76c3d" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 46641 } } peers { permanent_uuid: "40923b8b73cf438e98102a8d3b558db2" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 40871 } } peers { permanent_uuid: "6bfb041fcf764f73b5227cfee983bb3f" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 34537 } }
I20260806 22:50:08.704684 2479 leader_election.cc:288] T 90495bf3f6144afb9590ae87943161ac P 6bfb041fcf764f73b5227cfee983bb3f [CANDIDATE]: Term 1 election: Requested vote from peers 657f8e9000d2486586248a2d32e76c3d (127.28.218.193:46641), 40923b8b73cf438e98102a8d3b558db2 (127.28.218.194:40871)
I20260806 22:50:08.704825 2268 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90495bf3f6144afb9590ae87943161ac" candidate_uuid: "6bfb041fcf764f73b5227cfee983bb3f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40923b8b73cf438e98102a8d3b558db2"
I20260806 22:50:08.704842 2131 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90495bf3f6144afb9590ae87943161ac" candidate_uuid: "6bfb041fcf764f73b5227cfee983bb3f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657f8e9000d2486586248a2d32e76c3d"
I20260806 22:50:08.704908 2268 raft_consensus.cc:3037] T 90495bf3f6144afb9590ae87943161ac P 40923b8b73cf438e98102a8d3b558db2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:50:08.704929 2131 raft_consensus.cc:3037] T 90495bf3f6144afb9590ae87943161ac P 657f8e9000d2486586248a2d32e76c3d [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:50:08.705690 2131 raft_consensus.cc:2454] T 90495bf3f6144afb9590ae87943161ac P 657f8e9000d2486586248a2d32e76c3d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6bfb041fcf764f73b5227cfee983bb3f in term 1.
I20260806 22:50:08.705772 2268 raft_consensus.cc:2454] T 90495bf3f6144afb9590ae87943161ac P 40923b8b73cf438e98102a8d3b558db2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6bfb041fcf764f73b5227cfee983bb3f in term 1.
I20260806 22:50:08.705899 2336 leader_election.cc:302] T 90495bf3f6144afb9590ae87943161ac P 6bfb041fcf764f73b5227cfee983bb3f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 657f8e9000d2486586248a2d32e76c3d, 6bfb041fcf764f73b5227cfee983bb3f; no voters:
I20260806 22:50:08.706007 2479 raft_consensus.cc:2781] T 90495bf3f6144afb9590ae87943161ac P 6bfb041fcf764f73b5227cfee983bb3f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:50:08.706161 2479 raft_consensus.cc:686] T 90495bf3f6144afb9590ae87943161ac P 6bfb041fcf764f73b5227cfee983bb3f [term 1 LEADER]: Becoming Leader. State: Replica: 6bfb041fcf764f73b5227cfee983bb3f, State: Running, Role: LEADER
I20260806 22:50:08.706259 2479 consensus_queue.cc:237] T 90495bf3f6144afb9590ae87943161ac P 6bfb041fcf764f73b5227cfee983bb3f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "657f8e9000d2486586248a2d32e76c3d" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 46641 } } peers { permanent_uuid: "40923b8b73cf438e98102a8d3b558db2" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 40871 } } peers { permanent_uuid: "6bfb041fcf764f73b5227cfee983bb3f" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 34537 } }
I20260806 22:50:08.707209 1982 catalog_manager.cc:5495] T 90495bf3f6144afb9590ae87943161ac P 6bfb041fcf764f73b5227cfee983bb3f reported cstate change: term changed from 0 to 1, leader changed from <none> to 6bfb041fcf764f73b5227cfee983bb3f (127.28.218.195). New cstate: current_term: 1 leader_uuid: "6bfb041fcf764f73b5227cfee983bb3f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "657f8e9000d2486586248a2d32e76c3d" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 46641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "40923b8b73cf438e98102a8d3b558db2" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 40871 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6bfb041fcf764f73b5227cfee983bb3f" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 34537 } health_report { overall_health: HEALTHY } } }
I20260806 22:50:08.732887 2470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:08.732004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51486 (local address 127.28.218.195:34537)
0806 22:50:08.732057 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:08.732059 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:08.732067 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:50:08.732132 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:08.732132 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:08.732143 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:08.732190 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:08.732675 (+ 485us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:50:08.732676 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:50:08.732677 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:50:08.732680 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:50:08.732694 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:50:08.732698 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:50:08.732747 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:08.732780 (+ 33us) server_negotiation.cc:299] Negotiation successful
0806 22:50:08.732790 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 22:50:08.734608 2268 raft_consensus.cc:1261] T 90495bf3f6144afb9590ae87943161ac P 40923b8b73cf438e98102a8d3b558db2 [term 1 FOLLOWER]: Refusing update from remote peer 6bfb041fcf764f73b5227cfee983bb3f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:50:08.734681 2131 raft_consensus.cc:1261] T 90495bf3f6144afb9590ae87943161ac P 657f8e9000d2486586248a2d32e76c3d [term 1 FOLLOWER]: Refusing update from remote peer 6bfb041fcf764f73b5227cfee983bb3f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:50:08.734961 2483 consensus_queue.cc:1035] T 90495bf3f6144afb9590ae87943161ac P 6bfb041fcf764f73b5227cfee983bb3f [LEADER]: Connected to new peer: Peer: permanent_uuid: "40923b8b73cf438e98102a8d3b558db2" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 40871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:50:08.735087 2479 consensus_queue.cc:1035] T 90495bf3f6144afb9590ae87943161ac P 6bfb041fcf764f73b5227cfee983bb3f [LEADER]: Connected to new peer: Peer: permanent_uuid: "657f8e9000d2486586248a2d32e76c3d" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 46641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:50:08.737329 2487 mvcc.cc:205] Tried to move back new op lower bound from 7315687869374627840 to 7315687869261824000. Current Snapshot: MvccSnapshot[applied={T|T < 7315687869374627840}]
I20260806 22:50:08.738073 2488 mvcc.cc:205] Tried to move back new op lower bound from 7315687869374627840 to 7315687869261824000. Current Snapshot: MvccSnapshot[applied={T|T < 7315687869374627840}]
I20260806 22:50:08.738101 2489 mvcc.cc:205] Tried to move back new op lower bound from 7315687869374627840 to 7315687869261824000. Current Snapshot: MvccSnapshot[applied={T|T < 7315687869374627840}]
I20260806 22:50:08.739705 1982 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:51844:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:50:08.739800 1982 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:51844:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:50:08.740581 1982 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fc4c076baf1544a699451272bd9943bb: Sending DeleteTablet for 3 replicas of tablet 90495bf3f6144afb9590ae87943161ac
I20260806 22:50:08.740927 2110 tablet_service.cc:1514] Processing DeleteTablet for tablet 90495bf3f6144afb9590ae87943161ac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:50:08 UTC) from {username='slave'} at 127.0.0.1:46484
I20260806 22:50:08.740984 2387 tablet_service.cc:1514] Processing DeleteTablet for tablet 90495bf3f6144afb9590ae87943161ac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:50:08 UTC) from {username='slave'} at 127.0.0.1:51470
I20260806 22:50:08.741221 2246 tablet_service.cc:1514] Processing DeleteTablet for tablet 90495bf3f6144afb9590ae87943161ac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:50:08 UTC) from {username='slave'} at 127.0.0.1:51564
I20260806 22:50:08.741248 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 2045
I20260806 22:50:08.741369 2497 tablet_replica.cc:330] T 90495bf3f6144afb9590ae87943161ac P 657f8e9000d2486586248a2d32e76c3d: stopping tablet replica
I20260806 22:50:08.741464 2497 raft_consensus.cc:2229] T 90495bf3f6144afb9590ae87943161ac P 657f8e9000d2486586248a2d32e76c3d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:50:08.741555 2497 raft_consensus.cc:2258] T 90495bf3f6144afb9590ae87943161ac P 657f8e9000d2486586248a2d32e76c3d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:50:08.741786 2498 tablet_replica.cc:330] T 90495bf3f6144afb9590ae87943161ac P 6bfb041fcf764f73b5227cfee983bb3f: stopping tablet replica
I20260806 22:50:08.741866 2498 raft_consensus.cc:2229] T 90495bf3f6144afb9590ae87943161ac P 6bfb041fcf764f73b5227cfee983bb3f [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:50:08.741954 2498 raft_consensus.cc:2258] T 90495bf3f6144afb9590ae87943161ac P 6bfb041fcf764f73b5227cfee983bb3f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:50:08.742712 2498 ts_tablet_manager.cc:1905] T 90495bf3f6144afb9590ae87943161ac P 6bfb041fcf764f73b5227cfee983bb3f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:50:08.744349 2500 tablet_replica.cc:330] T 90495bf3f6144afb9590ae87943161ac P 40923b8b73cf438e98102a8d3b558db2: stopping tablet replica
I20260806 22:50:08.744521 2500 raft_consensus.cc:2229] T 90495bf3f6144afb9590ae87943161ac P 40923b8b73cf438e98102a8d3b558db2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:50:08.744531 2498 ts_tablet_manager.cc:1918] T 90495bf3f6144afb9590ae87943161ac P 6bfb041fcf764f73b5227cfee983bb3f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:50:08.744593 2498 log.cc:1196] T 90495bf3f6144afb9590ae87943161ac P 6bfb041fcf764f73b5227cfee983bb3f: Deleting WAL directory at /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056586788339-29547-0/minicluster-data/ts-2/wal/wals/90495bf3f6144afb9590ae87943161ac
I20260806 22:50:08.744726 2500 raft_consensus.cc:2258] T 90495bf3f6144afb9590ae87943161ac P 40923b8b73cf438e98102a8d3b558db2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:50:08.744894 2498 ts_tablet_manager.cc:1939] T 90495bf3f6144afb9590ae87943161ac P 6bfb041fcf764f73b5227cfee983bb3f: Deleting consensus metadata
I20260806 22:50:08.745339 1968 catalog_manager.cc:4841] TS 6bfb041fcf764f73b5227cfee983bb3f (127.28.218.195:34537): tablet 90495bf3f6144afb9590ae87943161ac (table test-table [id=73607dda279549f9bc7847bd14cf4ba7]) successfully deleted
W20260806 22:50:08.748853 1967 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.28.218.193:46641 (error 108)
I20260806 22:50:08.749051 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 2181
I20260806 22:50:08.749336 2466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:08.749217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.193:46641 (local address 127.0.0.1:46486)
0806 22:50:08.749265 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:08.749303 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.193:46641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260806 22:50:08.756126 1967 catalog_manager.cc:4584] TS 657f8e9000d2486586248a2d32e76c3d (127.28.218.193:46641): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 90495bf3f6144afb9590ae87943161ac: Network error: Client connection negotiation failed: client connection to 127.28.218.193:46641: connect: Connection refused (error 111)
I20260806 22:50:08.757345 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 2317
I20260806 22:50:08.757364 2466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:08.757259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.194:40871 (local address 127.0.0.1:51580)
0806 22:50:08.757305 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:08.757327 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.194:40871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260806 22:50:08.762169 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 1948
2026-08-06T22:50:08Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (694 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260806 22:50:08.783756 29547 mini_oidc.cc:152] Starting JWKS server
I20260806 22:50:08.784888 29547 webserver.cc:460] Webserver started at https://127.0.0.1:35075/ using document root <none> and password file <none>
I20260806 22:50:08.784999 29547 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 22:50:08.785168 29547 webserver.cc:460] Webserver started at http://127.0.0.1:44365/ using document root <none> and password file <none>
2026-08-06T22:50:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:50:08Z Disabled control of system clock
I20260806 22:50:08.793816 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786056586788339-29547-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786056586788339-29547-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786056586788339-29547-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.218.254:46063
--webserver_interface=127.28.218.254
--webserver_port=0
--builtin_ntp_servers=127.28.218.212:34073
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.218.254:46063
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35075/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786056586788339-29547-0/testchainca.cert with env {}
W20260806 22:50:08.874408 2507 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:50:08.874591 2507 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:50:08.874614 2507 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:50:08.875779 2507 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:50:08.875820 2507 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:50:08.875833 2507 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:50:08.875875 2507 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:50:08.875901 2507 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 22:50:08.875917 2507 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 22:50:08.875943 2507 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:35075/jwks.json
I20260806 22:50:08.877527 2507 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:34073
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786056586788339-29547-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786056586788339-29547-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.218.254:46063
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.218.254:46063
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35075/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.218.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786056586788339-29547-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2507
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786056586788339-29547-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:50:08.877776 2507 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:50:08.877995 2507 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:50:08.882714 2515 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:50:08.882788 2507 server_base.cc:1034] running on GCE node
W20260806 22:50:08.882722 2513 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:08.882706 2512 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:50:08.884023 2507 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:50:08.884291 2507 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:50:08.885442 2507 hybrid_clock.cc:648] HybridClock initialized: now 1786056608885435 us; error 45 us; skew 500 ppm
I20260806 22:50:08.886615 2507 webserver.cc:460] Webserver started at http://127.28.218.254:35845/ using document root <none> and password file <none>
I20260806 22:50:08.886834 2507 fs_manager.cc:359] Metadata directory not provided
I20260806 22:50:08.886909 2507 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:50:08.887044 2507 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:50:08.888005 2507 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786056586788339-29547-0/minicluster-data/master-0/data/instance:
uuid: "5796c61a7bc24a588a0aabf3191120cc"
format_stamp: "Formatted at 2026-08-06 22:50:08 on dist-test-slave-1xrh"
server_key: "7f2f7ce0cf239933ed046e8ef54f7c79"
server_key_iv: "2448fe8f3790019b86568c27289b5683"
server_key_version: "encryptionkey@0"
I20260806 22:50:08.888338 2507 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786056586788339-29547-0/minicluster-data/master-0/wal/instance:
uuid: "5796c61a7bc24a588a0aabf3191120cc"
format_stamp: "Formatted at 2026-08-06 22:50:08 on dist-test-slave-1xrh"
server_key: "7f2f7ce0cf239933ed046e8ef54f7c79"
server_key_iv: "2448fe8f3790019b86568c27289b5683"
server_key_version: "encryptionkey@0"
I20260806 22:50:08.889524 2507 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 22:50:08.890161 2522 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:08.890318 2507 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:50:08.890407 2507 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786056586788339-29547-0/minicluster-data/master-0/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786056586788339-29547-0/minicluster-data/master-0/wal
uuid: "5796c61a7bc24a588a0aabf3191120cc"
format_stamp: "Formatted at 2026-08-06 22:50:08 on dist-test-slave-1xrh"
server_key: "7f2f7ce0cf239933ed046e8ef54f7c79"
server_key_iv: "2448fe8f3790019b86568c27289b5683"
server_key_version: "encryptionkey@0"
I20260806 22:50:08.890494 2507 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786056586788339-29547-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786056586788339-29547-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786056586788339-29547-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:50:08.909948 2507 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:50:08.910202 2507 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:50:08.910336 2507 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:50:08.914077 2507 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.254:46063
I20260806 22:50:08.914113 2576 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.254:46063 every 8 connection(s)
I20260806 22:50:08.914422 2507 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
I20260806 22:50:08.915514 2577 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:50:08.917667 2577 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5796c61a7bc24a588a0aabf3191120cc: Bootstrap starting.
I20260806 22:50:08.918061 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 2507
I20260806 22:50:08.918148 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786056586788339-29547-0/minicluster-data/master-0/wal/instance
I20260806 22:50:08.918334 29547 external_mini_cluster.cc:1428] Setting key 550556cae509b319c72e44a4df655653
I20260806 22:50:08.918337 2577 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5796c61a7bc24a588a0aabf3191120cc: Neither blocks nor log segments found. Creating new log.
I20260806 22:50:08.918707 2577 log.cc:824] T 00000000000000000000000000000000 P 5796c61a7bc24a588a0aabf3191120cc: Log is configured to *not* fsync() on all Append() calls
I20260806 22:50:08.919528 2577 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5796c61a7bc24a588a0aabf3191120cc: No bootstrap required, opened a new log
I20260806 22:50:08.922010 2577 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5796c61a7bc24a588a0aabf3191120cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5796c61a7bc24a588a0aabf3191120cc" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 46063 } }
I20260806 22:50:08.922171 2577 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5796c61a7bc24a588a0aabf3191120cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:50:08.922202 2577 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5796c61a7bc24a588a0aabf3191120cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5796c61a7bc24a588a0aabf3191120cc, State: Initialized, Role: FOLLOWER
I20260806 22:50:08.922339 2577 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5796c61a7bc24a588a0aabf3191120cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5796c61a7bc24a588a0aabf3191120cc" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 46063 } }
I20260806 22:50:08.922436 2577 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5796c61a7bc24a588a0aabf3191120cc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:50:08.922483 2577 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5796c61a7bc24a588a0aabf3191120cc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:50:08.922538 2577 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5796c61a7bc24a588a0aabf3191120cc [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:50:08.923305 2577 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5796c61a7bc24a588a0aabf3191120cc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5796c61a7bc24a588a0aabf3191120cc" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 46063 } }
I20260806 22:50:08.923427 2577 leader_election.cc:302] T 00000000000000000000000000000000 P 5796c61a7bc24a588a0aabf3191120cc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5796c61a7bc24a588a0aabf3191120cc; no voters:
I20260806 22:50:08.923614 2577 leader_election.cc:288] T 00000000000000000000000000000000 P 5796c61a7bc24a588a0aabf3191120cc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:50:08.923659 2582 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5796c61a7bc24a588a0aabf3191120cc [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:50:08.923795 2582 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5796c61a7bc24a588a0aabf3191120cc [term 1 LEADER]: Becoming Leader. State: Replica: 5796c61a7bc24a588a0aabf3191120cc, State: Running, Role: LEADER
I20260806 22:50:08.923897 2577 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5796c61a7bc24a588a0aabf3191120cc [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:50:08.923902 2582 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5796c61a7bc24a588a0aabf3191120cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5796c61a7bc24a588a0aabf3191120cc" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 46063 } }
I20260806 22:50:08.924731 2584 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5796c61a7bc24a588a0aabf3191120cc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5796c61a7bc24a588a0aabf3191120cc. Latest consensus state: current_term: 1 leader_uuid: "5796c61a7bc24a588a0aabf3191120cc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5796c61a7bc24a588a0aabf3191120cc" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 46063 } } }
I20260806 22:50:08.924824 2584 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5796c61a7bc24a588a0aabf3191120cc [sys.catalog]: This master's current role is: LEADER
I20260806 22:50:08.924904 2580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:08.918788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36322 (local address 127.28.218.254:46063)
0806 22:50:08.918970 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:08.918976 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:08.919001 (+ 25us) server_negotiation.cc:407] Connection header received
0806 22:50:08.919529 (+ 528us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:08.919537 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:08.919619 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:08.919765 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:08.919933 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:08.921270 (+ 1337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:08.922517 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:08.923209 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:08.923349 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:08.924064 (+ 715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:50:08.924068 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:50:08.924068 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:50:08.924263 (+ 195us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:50:08.924294 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:50:08.924304 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:50:08.924383 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:08.924580 (+ 197us) server_negotiation.cc:299] Negotiation successful
0806 22:50:08.924668 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":72,"threads_started":1}
I20260806 22:50:08.925531 2583 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5796c61a7bc24a588a0aabf3191120cc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5796c61a7bc24a588a0aabf3191120cc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5796c61a7bc24a588a0aabf3191120cc" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 46063 } } }
I20260806 22:50:08.925642 2583 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5796c61a7bc24a588a0aabf3191120cc [sys.catalog]: This master's current role is: LEADER
W20260806 22:50:08.925836 2598 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5796c61a7bc24a588a0aabf3191120cc: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 22:50:08.925877 2598 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 22:50:08.925873 29547 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:50:08.925947 2591 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:50:08.926092 2591 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:50:08.927433 2591 catalog_manager.cc:1347] Generated new cluster ID: 236a9f4cddf44107a3b30973572ec11b
I20260806 22:50:08.927486 2591 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:50:08.931303 2591 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:50:08.931819 2591 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:50:08.935776 2591 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5796c61a7bc24a588a0aabf3191120cc: Generated new TSK 0
I20260806 22:50:08.935925 2591 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260806 22:50:08.967278 2608 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260806 22:50:08.967507 2608 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260806 22:50:08.967546 2580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:08.944560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36338 (local address 127.28.218.254:46063)
0806 22:50:08.944614 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:08.944617 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:08.944742 (+ 125us) server_negotiation.cc:407] Connection header received
0806 22:50:08.944836 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:08.944837 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:08.944846 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:08.944894 (+ 48us) server_negotiation.cc:226] Negotiated authn=JWT
0806 22:50:08.945044 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:08.945502 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:08.946681 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:08.946857 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:08.946904 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:08.947110 (+ 206us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 22:50:08.967275 (+ 20165us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35075/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0806 22:50:08.967450 (+ 175us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:36338: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35075/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":32}
W20260806 22:50:08.967604 2580 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:36338: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35075/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260806 22:50:08.967844 2607 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:50:08.944506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:46063 (local address 127.0.0.1:36338)
0806 22:50:08.944675 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:08.944687 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:08.944753 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:08.944909 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:08.944910 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:08.944912 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0806 22:50:08.944976 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:08.944979 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:08.945596 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:08.945598 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:08.946606 (+ 1008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:08.946608 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:08.946933 (+ 325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:08.946934 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:08.946997 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:08.947018 (+ 21us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 22:50:08.967415 (+ 20397us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35075/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0806 22:50:08.967530 (+ 115us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.28.218.254:46063: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35075/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":90,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.28.218.254:46063: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35075/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260806 22:50:08.968195 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 2507
2026-08-06T22:50:08Z chronyd exiting
I20260806 22:50:09.004922 29547 test_util.cc:175] -----------------------------------------------
I20260806 22:50:09.005008 29547 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786056586788339-29547-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (222 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260806 22:50:09.005752 29547 mini_oidc.cc:152] Starting JWKS server
I20260806 22:50:09.006425 29547 webserver.cc:460] Webserver started at https://127.0.0.1:35219/ using document root <none> and password file <none>
I20260806 22:50:09.006490 29547 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 22:50:09.006662 29547 webserver.cc:460] Webserver started at http://127.0.0.1:46089/ using document root <none> and password file <none>
2026-08-06T22:50:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:50:09Z Disabled control of system clock
I20260806 22:50:09.014771 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786056586788339-29547-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786056586788339-29547-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786056586788339-29547-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.218.254:39183
--webserver_interface=127.28.218.254
--webserver_port=0
--builtin_ntp_servers=127.28.218.212:45309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.218.254:39183
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35219/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786056586788339-29547-0/testchainca.cert with env {}
W20260806 22:50:09.093771 2614 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:50:09.093955 2614 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:50:09.093976 2614 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:50:09.095005 2614 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:50:09.095039 2614 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:50:09.095052 2614 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:50:09.095064 2614 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:50:09.095077 2614 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 22:50:09.095088 2614 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 22:50:09.095099 2614 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:35219/jwks.json
I20260806 22:50:09.096608 2614 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:45309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786056586788339-29547-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786056586788339-29547-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.218.254:39183
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.218.254:39183
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35219/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.218.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786056586788339-29547-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2614
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786056586788339-29547-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:50:09.096796 2614 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:50:09.096968 2614 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:50:09.101490 2619 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:09.101572 2622 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:09.101624 2620 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:50:09.101871 2614 server_base.cc:1034] running on GCE node
I20260806 22:50:09.102108 2614 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:50:09.102368 2614 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:50:09.103518 2614 hybrid_clock.cc:648] HybridClock initialized: now 1786056609103511 us; error 40 us; skew 500 ppm
I20260806 22:50:09.104713 2614 webserver.cc:460] Webserver started at http://127.28.218.254:34907/ using document root <none> and password file <none>
I20260806 22:50:09.104941 2614 fs_manager.cc:359] Metadata directory not provided
I20260806 22:50:09.105010 2614 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:50:09.105158 2614 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:50:09.106141 2614 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786056586788339-29547-0/minicluster-data/master-0/data/instance:
uuid: "2a15c82286df4475be9d6e50b9788750"
format_stamp: "Formatted at 2026-08-06 22:50:09 on dist-test-slave-1xrh"
server_key: "f745ddde5649636897f71ed508b5a905"
server_key_iv: "dc98dc330e4bf2a7411c8afc1d4d8666"
server_key_version: "encryptionkey@0"
I20260806 22:50:09.106488 2614 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786056586788339-29547-0/minicluster-data/master-0/wal/instance:
uuid: "2a15c82286df4475be9d6e50b9788750"
format_stamp: "Formatted at 2026-08-06 22:50:09 on dist-test-slave-1xrh"
server_key: "f745ddde5649636897f71ed508b5a905"
server_key_iv: "dc98dc330e4bf2a7411c8afc1d4d8666"
server_key_version: "encryptionkey@0"
I20260806 22:50:09.107797 2614 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:50:09.108548 2631 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:09.108752 2614 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:50:09.108846 2614 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786056586788339-29547-0/minicluster-data/master-0/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786056586788339-29547-0/minicluster-data/master-0/wal
uuid: "2a15c82286df4475be9d6e50b9788750"
format_stamp: "Formatted at 2026-08-06 22:50:09 on dist-test-slave-1xrh"
server_key: "f745ddde5649636897f71ed508b5a905"
server_key_iv: "dc98dc330e4bf2a7411c8afc1d4d8666"
server_key_version: "encryptionkey@0"
I20260806 22:50:09.108902 2614 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786056586788339-29547-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786056586788339-29547-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786056586788339-29547-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:50:09.123039 2614 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:50:09.123358 2614 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:50:09.123467 2614 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:50:09.127524 2614 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.254:39183
I20260806 22:50:09.127553 2687 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.254:39183 every 8 connection(s)
I20260806 22:50:09.127903 2614 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
I20260806 22:50:09.128557 2688 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:50:09.128978 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 2614
I20260806 22:50:09.129127 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786056586788339-29547-0/minicluster-data/master-0/wal/instance
I20260806 22:50:09.129277 29547 external_mini_cluster.cc:1428] Setting key dd6ff7f47c634942bddd34ff229f832f
I20260806 22:50:09.132104 2688 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a15c82286df4475be9d6e50b9788750: Bootstrap starting.
I20260806 22:50:09.132757 2688 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2a15c82286df4475be9d6e50b9788750: Neither blocks nor log segments found. Creating new log.
I20260806 22:50:09.133042 2688 log.cc:824] T 00000000000000000000000000000000 P 2a15c82286df4475be9d6e50b9788750: Log is configured to *not* fsync() on all Append() calls
I20260806 22:50:09.133663 2688 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a15c82286df4475be9d6e50b9788750: No bootstrap required, opened a new log
I20260806 22:50:09.134747 2691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:09.129766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48206 (local address 127.28.218.254:39183)
0806 22:50:09.129950 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:09.129957 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:09.129981 (+ 24us) server_negotiation.cc:407] Connection header received
0806 22:50:09.130492 (+ 511us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:09.130498 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:09.130554 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:09.130662 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:09.130894 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:09.131773 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:09.132853 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:09.133319 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:09.133448 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:09.134136 (+ 688us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:50:09.134140 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:50:09.134140 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:50:09.134296 (+ 156us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:50:09.134325 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:50:09.134334 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:50:09.134405 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:09.134488 (+ 83us) server_negotiation.cc:299] Negotiation successful
0806 22:50:09.134567 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":87,"threads_started":1}
I20260806 22:50:09.135460 2688 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2a15c82286df4475be9d6e50b9788750 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a15c82286df4475be9d6e50b9788750" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 39183 } }
I20260806 22:50:09.135640 2688 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2a15c82286df4475be9d6e50b9788750 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:50:09.135689 2688 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2a15c82286df4475be9d6e50b9788750 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a15c82286df4475be9d6e50b9788750, State: Initialized, Role: FOLLOWER
I20260806 22:50:09.135742 29547 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:50:09.135828 2688 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2a15c82286df4475be9d6e50b9788750 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a15c82286df4475be9d6e50b9788750" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 39183 } }
I20260806 22:50:09.135900 2688 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2a15c82286df4475be9d6e50b9788750 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:50:09.135927 2688 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2a15c82286df4475be9d6e50b9788750 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:50:09.135977 2688 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2a15c82286df4475be9d6e50b9788750 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:50:09.136618 2688 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2a15c82286df4475be9d6e50b9788750 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a15c82286df4475be9d6e50b9788750" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 39183 } }
I20260806 22:50:09.136734 2688 leader_election.cc:302] T 00000000000000000000000000000000 P 2a15c82286df4475be9d6e50b9788750 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2a15c82286df4475be9d6e50b9788750; no voters:
I20260806 22:50:09.136895 2688 leader_election.cc:288] T 00000000000000000000000000000000 P 2a15c82286df4475be9d6e50b9788750 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:50:09.136987 2693 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2a15c82286df4475be9d6e50b9788750 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:50:09.137112 2688 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2a15c82286df4475be9d6e50b9788750 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:50:09.137141 2693 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2a15c82286df4475be9d6e50b9788750 [term 1 LEADER]: Becoming Leader. State: Replica: 2a15c82286df4475be9d6e50b9788750, State: Running, Role: LEADER
I20260806 22:50:09.137216 2693 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2a15c82286df4475be9d6e50b9788750 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a15c82286df4475be9d6e50b9788750" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 39183 } }
I20260806 22:50:09.137594 2694 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a15c82286df4475be9d6e50b9788750 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2a15c82286df4475be9d6e50b9788750" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a15c82286df4475be9d6e50b9788750" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 39183 } } }
I20260806 22:50:09.137610 2695 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a15c82286df4475be9d6e50b9788750 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2a15c82286df4475be9d6e50b9788750. Latest consensus state: current_term: 1 leader_uuid: "2a15c82286df4475be9d6e50b9788750" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a15c82286df4475be9d6e50b9788750" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 39183 } } }
I20260806 22:50:09.137744 2694 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a15c82286df4475be9d6e50b9788750 [sys.catalog]: This master's current role is: LEADER
I20260806 22:50:09.137758 2695 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a15c82286df4475be9d6e50b9788750 [sys.catalog]: This master's current role is: LEADER
I20260806 22:50:09.138209 2703 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:50:09.138688 2703 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:50:09.140085 2703 catalog_manager.cc:1347] Generated new cluster ID: c70e7591029846df9ea6a4305e298fde
I20260806 22:50:09.140136 2703 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:50:09.149405 2703 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:50:09.149818 2703 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:50:09.163705 2703 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2a15c82286df4475be9d6e50b9788750: Generated new TSK 0
I20260806 22:50:09.163841 2703 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260806 22:50:09.188635 2720 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260806 22:50:09.188709 2720 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260806 22:50:09.188997 2691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:09.175103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48222 (local address 127.28.218.254:39183)
0806 22:50:09.175159 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:09.175161 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:09.175294 (+ 133us) server_negotiation.cc:407] Connection header received
0806 22:50:09.175333 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:09.175334 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:09.175347 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:09.175430 (+ 83us) server_negotiation.cc:226] Negotiated authn=JWT
0806 22:50:09.175541 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:09.176321 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:09.177056 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:09.177405 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:09.177485 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:09.177618 (+ 133us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 22:50:09.188789 (+ 11171us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35219/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0806 22:50:09.188933 (+ 144us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:48222: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35219/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":38}
W20260806 22:50:09.189050 2691 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:48222: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35219/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260806 22:50:09.189057 2719 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:50:09.175059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:39183 (local address 127.0.0.1:48222)
0806 22:50:09.175228 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:09.175238 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:09.175291 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:09.175428 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:09.175428 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:09.175430 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0806 22:50:09.175488 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:09.175490 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:09.176392 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:09.176393 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:09.176975 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:09.176976 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:09.177493 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:09.177494 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:09.177534 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:09.177538 (+ 4us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 22:50:09.188880 (+ 11342us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35219/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0806 22:50:09.188978 (+ 98us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.28.218.254:39183: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35219/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":113,"threads_started":1}
I20260806 22:50:09.189482 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 2614
2026-08-06T22:50:09Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (222 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260806 22:50:09.228430 29547 mini_oidc.cc:152] Starting JWKS server
I20260806 22:50:09.229065 29547 webserver.cc:460] Webserver started at https://127.0.0.1:43827/ using document root <none> and password file <none>
I20260806 22:50:09.229115 29547 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 22:50:09.229247 29547 webserver.cc:460] Webserver started at http://127.0.0.1:33707/ using document root <none> and password file <none>
2026-08-06T22:50:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:50:09Z Disabled control of system clock
I20260806 22:50:09.237596 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786056586788339-29547-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786056586788339-29547-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786056586788339-29547-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.218.254:45205
--webserver_interface=127.28.218.254
--webserver_port=0
--builtin_ntp_servers=127.28.218.212:43135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.218.254:45205
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43827/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260806 22:50:09.316644 2726 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:50:09.316900 2726 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:50:09.316933 2726 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:50:09.318593 2726 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:50:09.318646 2726 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:50:09.318666 2726 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:50:09.318686 2726 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:50:09.318706 2726 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 22:50:09.318734 2726 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 22:50:09.318763 2726 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:43827/jwks.json
I20260806 22:50:09.320530 2726 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:43135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786056586788339-29547-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786056586788339-29547-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.218.254:45205
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.218.254:45205
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43827/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.218.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2726
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786056586788339-29547-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:50:09.320739 2726 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:50:09.320952 2726 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:50:09.325974 2735 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:09.325958 2732 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:09.325944 2733 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:50:09.326275 2726 server_base.cc:1034] running on GCE node
I20260806 22:50:09.326493 2726 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:50:09.326726 2726 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:50:09.327890 2726 hybrid_clock.cc:648] HybridClock initialized: now 1786056609327854 us; error 49 us; skew 500 ppm
I20260806 22:50:09.329185 2726 webserver.cc:460] Webserver started at http://127.28.218.254:44915/ using document root <none> and password file <none>
I20260806 22:50:09.329429 2726 fs_manager.cc:359] Metadata directory not provided
I20260806 22:50:09.329478 2726 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:50:09.329602 2726 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:50:09.330657 2726 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786056586788339-29547-0/minicluster-data/master-0/data/instance:
uuid: "7fdfc9bb9fa34f51a3352429c84fcf77"
format_stamp: "Formatted at 2026-08-06 22:50:09 on dist-test-slave-1xrh"
server_key: "0e5f890096b167ff0c00c32a46993a1b"
server_key_iv: "b06c106d4bedc0203f97e46fb86e7981"
server_key_version: "encryptionkey@0"
I20260806 22:50:09.331015 2726 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786056586788339-29547-0/minicluster-data/master-0/wal/instance:
uuid: "7fdfc9bb9fa34f51a3352429c84fcf77"
format_stamp: "Formatted at 2026-08-06 22:50:09 on dist-test-slave-1xrh"
server_key: "0e5f890096b167ff0c00c32a46993a1b"
server_key_iv: "b06c106d4bedc0203f97e46fb86e7981"
server_key_version: "encryptionkey@0"
I20260806 22:50:09.332312 2726 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:50:09.332989 2742 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:09.333146 2726 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:50:09.333241 2726 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786056586788339-29547-0/minicluster-data/master-0/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786056586788339-29547-0/minicluster-data/master-0/wal
uuid: "7fdfc9bb9fa34f51a3352429c84fcf77"
format_stamp: "Formatted at 2026-08-06 22:50:09 on dist-test-slave-1xrh"
server_key: "0e5f890096b167ff0c00c32a46993a1b"
server_key_iv: "b06c106d4bedc0203f97e46fb86e7981"
server_key_version: "encryptionkey@0"
I20260806 22:50:09.333310 2726 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786056586788339-29547-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786056586788339-29547-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786056586788339-29547-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:50:09.347527 2726 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:50:09.347792 2726 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:50:09.347941 2726 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:50:09.351660 2726 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.254:45205
I20260806 22:50:09.351670 2796 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.254:45205 every 8 connection(s)
I20260806 22:50:09.352032 2726 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
I20260806 22:50:09.352690 2797 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:50:09.355259 2797 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7fdfc9bb9fa34f51a3352429c84fcf77: Bootstrap starting.
I20260806 22:50:09.355767 2797 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7fdfc9bb9fa34f51a3352429c84fcf77: Neither blocks nor log segments found. Creating new log.
I20260806 22:50:09.355995 2797 log.cc:824] T 00000000000000000000000000000000 P 7fdfc9bb9fa34f51a3352429c84fcf77: Log is configured to *not* fsync() on all Append() calls
I20260806 22:50:09.356633 2797 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7fdfc9bb9fa34f51a3352429c84fcf77: No bootstrap required, opened a new log
I20260806 22:50:09.358182 2797 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7fdfc9bb9fa34f51a3352429c84fcf77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fdfc9bb9fa34f51a3352429c84fcf77" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 45205 } }
I20260806 22:50:09.358307 2797 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7fdfc9bb9fa34f51a3352429c84fcf77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:50:09.358330 2797 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7fdfc9bb9fa34f51a3352429c84fcf77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fdfc9bb9fa34f51a3352429c84fcf77, State: Initialized, Role: FOLLOWER
I20260806 22:50:09.358453 2797 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7fdfc9bb9fa34f51a3352429c84fcf77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fdfc9bb9fa34f51a3352429c84fcf77" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 45205 } }
I20260806 22:50:09.358537 2797 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7fdfc9bb9fa34f51a3352429c84fcf77 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:50:09.358560 2797 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7fdfc9bb9fa34f51a3352429c84fcf77 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:50:09.358588 2797 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7fdfc9bb9fa34f51a3352429c84fcf77 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:50:09.359073 2797 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7fdfc9bb9fa34f51a3352429c84fcf77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fdfc9bb9fa34f51a3352429c84fcf77" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 45205 } }
I20260806 22:50:09.359215 2797 leader_election.cc:302] T 00000000000000000000000000000000 P 7fdfc9bb9fa34f51a3352429c84fcf77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7fdfc9bb9fa34f51a3352429c84fcf77; no voters:
I20260806 22:50:09.359338 2797 leader_election.cc:288] T 00000000000000000000000000000000 P 7fdfc9bb9fa34f51a3352429c84fcf77 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:50:09.359412 2801 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7fdfc9bb9fa34f51a3352429c84fcf77 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:50:09.359555 2801 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7fdfc9bb9fa34f51a3352429c84fcf77 [term 1 LEADER]: Becoming Leader. State: Replica: 7fdfc9bb9fa34f51a3352429c84fcf77, State: Running, Role: LEADER
I20260806 22:50:09.359589 2797 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7fdfc9bb9fa34f51a3352429c84fcf77 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:50:09.359647 2801 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7fdfc9bb9fa34f51a3352429c84fcf77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fdfc9bb9fa34f51a3352429c84fcf77" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 45205 } }
I20260806 22:50:09.359984 2802 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7fdfc9bb9fa34f51a3352429c84fcf77 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7fdfc9bb9fa34f51a3352429c84fcf77" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fdfc9bb9fa34f51a3352429c84fcf77" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 45205 } } }
I20260806 22:50:09.360091 2802 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7fdfc9bb9fa34f51a3352429c84fcf77 [sys.catalog]: This master's current role is: LEADER
I20260806 22:50:09.360149 2803 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7fdfc9bb9fa34f51a3352429c84fcf77 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7fdfc9bb9fa34f51a3352429c84fcf77. Latest consensus state: current_term: 1 leader_uuid: "7fdfc9bb9fa34f51a3352429c84fcf77" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fdfc9bb9fa34f51a3352429c84fcf77" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 45205 } } }
I20260806 22:50:09.360234 2803 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7fdfc9bb9fa34f51a3352429c84fcf77 [sys.catalog]: This master's current role is: LEADER
I20260806 22:50:09.360388 2809 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:50:09.360879 2809 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:50:09.361937 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 2726
I20260806 22:50:09.362059 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786056586788339-29547-0/minicluster-data/master-0/wal/instance
I20260806 22:50:09.362272 29547 external_mini_cluster.cc:1428] Setting key 2475a32abc9b4dd5262ae9006cb31031
I20260806 22:50:09.362375 2809 catalog_manager.cc:1347] Generated new cluster ID: df0bdb6838724497b8e95b4124f4fad4
I20260806 22:50:09.362423 2809 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:50:09.368592 2821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:09.362684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51124 (local address 127.28.218.254:45205)
0806 22:50:09.363045 (+ 361us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:09.363053 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:09.363080 (+ 27us) server_negotiation.cc:407] Connection header received
0806 22:50:09.363218 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:09.363224 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:09.363286 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:09.363401 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:09.363637 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:09.364986 (+ 1349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:09.366097 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:09.366667 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:09.366769 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:09.367629 (+ 860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:50:09.367632 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:50:09.367633 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:50:09.367850 (+ 217us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:50:09.367895 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:50:09.367909 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:50:09.367991 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:09.368051 (+ 60us) server_negotiation.cc:299] Negotiation successful
0806 22:50:09.368125 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":291,"thread_start_us":73,"threads_started":1}
I20260806 22:50:09.369560 29547 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:50:09.376145 2809 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:50:09.376605 2809 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:50:09.378933 2809 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7fdfc9bb9fa34f51a3352429c84fcf77: Generated new TSK 0
I20260806 22:50:09.379096 2809 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:50:09.386929 2821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:09.383362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51132 (local address 127.28.218.254:45205)
0806 22:50:09.383403 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:09.383405 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:09.383512 (+ 107us) server_negotiation.cc:407] Connection header received
0806 22:50:09.383586 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:09.383586 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:09.383595 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:09.383644 (+ 49us) server_negotiation.cc:226] Negotiated authn=JWT
0806 22:50:09.383768 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:09.384246 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:09.385390 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:09.385593 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:09.385641 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:09.385830 (+ 189us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 22:50:09.386717 (+ 887us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43827/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0806 22:50:09.386858 (+ 141us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:51132: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43827/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":23}
W20260806 22:50:09.386987 2821 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:51132: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43827/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260806 22:50:09.387015 2828 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:50:09.383305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:45205 (local address 127.0.0.1:51132)
0806 22:50:09.383451 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:09.383462 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:09.383524 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:09.383654 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:09.383654 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:09.383656 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0806 22:50:09.383715 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:09.383717 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:09.384346 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:09.384347 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:09.385296 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:09.385299 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:09.385665 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:09.385666 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:09.385730 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:09.385735 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 22:50:09.386820 (+ 1085us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43827/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0806 22:50:09.386905 (+ 85us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.28.218.254:45205: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43827/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":76,"threads_started":1}
I20260806 22:50:09.387413 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 2726
E20260806 22:50:09.399931 2829 webserver.cc:533] Webserver: sslize failed:
E20260806 22:50:09.399994 2829 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-06T22:50:09Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (198 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-06T22:50:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:50:09Z Disabled control of system clock
I20260806 22:50:09.434943 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786056586788339-29547-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786056586788339-29547-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786056586788339-29547-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.218.254:35415
--webserver_interface=127.28.218.254
--webserver_port=0
--builtin_ntp_servers=127.28.218.212:37665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.218.254:35415
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786056586788339-29547-0/testchainca.cert with env {}
W20260806 22:50:09.514241 2835 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:50:09.514411 2835 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:50:09.514431 2835 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:50:09.515623 2835 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:50:09.515663 2835 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:50:09.515676 2835 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:50:09.515707 2835 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:50:09.515738 2835 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 22:50:09.515779 2835 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 22:50:09.515807 2835 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260806 22:50:09.517259 2835 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:37665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786056586788339-29547-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786056586788339-29547-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.218.254:35415
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.218.254:35415
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.218.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786056586788339-29547-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2835
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786056586788339-29547-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:50:09.517485 2835 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:50:09.517716 2835 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:50:09.522330 2840 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:09.522343 2843 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:09.522394 2841 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:50:09.522558 2835 server_base.cc:1034] running on GCE node
I20260806 22:50:09.522791 2835 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:50:09.523077 2835 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:50:09.524251 2835 hybrid_clock.cc:648] HybridClock initialized: now 1786056609524234 us; error 35 us; skew 500 ppm
I20260806 22:50:09.525426 2835 webserver.cc:460] Webserver started at http://127.28.218.254:45879/ using document root <none> and password file <none>
I20260806 22:50:09.525667 2835 fs_manager.cc:359] Metadata directory not provided
I20260806 22:50:09.525735 2835 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:50:09.525866 2835 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:50:09.526872 2835 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786056586788339-29547-0/minicluster-data/master-0/data/instance:
uuid: "5d55b3dfb4fc432ba58190a1994187b9"
format_stamp: "Formatted at 2026-08-06 22:50:09 on dist-test-slave-1xrh"
server_key: "4a7b9839e697b91edf710437b92b6a23"
server_key_iv: "f022a31e8172c97f42695815f058faf0"
server_key_version: "encryptionkey@0"
I20260806 22:50:09.527271 2835 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786056586788339-29547-0/minicluster-data/master-0/wal/instance:
uuid: "5d55b3dfb4fc432ba58190a1994187b9"
format_stamp: "Formatted at 2026-08-06 22:50:09 on dist-test-slave-1xrh"
server_key: "4a7b9839e697b91edf710437b92b6a23"
server_key_iv: "f022a31e8172c97f42695815f058faf0"
server_key_version: "encryptionkey@0"
I20260806 22:50:09.528537 2835 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:50:09.529258 2851 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:09.529532 2835 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:50:09.529632 2835 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786056586788339-29547-0/minicluster-data/master-0/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786056586788339-29547-0/minicluster-data/master-0/wal
uuid: "5d55b3dfb4fc432ba58190a1994187b9"
format_stamp: "Formatted at 2026-08-06 22:50:09 on dist-test-slave-1xrh"
server_key: "4a7b9839e697b91edf710437b92b6a23"
server_key_iv: "f022a31e8172c97f42695815f058faf0"
server_key_version: "encryptionkey@0"
I20260806 22:50:09.529698 2835 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786056586788339-29547-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786056586788339-29547-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786056586788339-29547-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:50:09.539398 2835 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:50:09.539664 2835 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:50:09.539801 2835 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:50:09.543658 2835 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.254:35415
I20260806 22:50:09.543689 2905 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.254:35415 every 8 connection(s)
I20260806 22:50:09.544118 2835 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
I20260806 22:50:09.545261 2906 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:50:09.547500 2906 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d55b3dfb4fc432ba58190a1994187b9: Bootstrap starting.
I20260806 22:50:09.548053 2906 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5d55b3dfb4fc432ba58190a1994187b9: Neither blocks nor log segments found. Creating new log.
I20260806 22:50:09.548319 2906 log.cc:824] T 00000000000000000000000000000000 P 5d55b3dfb4fc432ba58190a1994187b9: Log is configured to *not* fsync() on all Append() calls
I20260806 22:50:09.548950 2906 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d55b3dfb4fc432ba58190a1994187b9: No bootstrap required, opened a new log
I20260806 22:50:09.549203 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 2835
I20260806 22:50:09.549309 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786056586788339-29547-0/minicluster-data/master-0/wal/instance
I20260806 22:50:09.549494 29547 external_mini_cluster.cc:1428] Setting key 6051b213ccbd9334f55b2e1d93014009
I20260806 22:50:09.551294 2906 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5d55b3dfb4fc432ba58190a1994187b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d55b3dfb4fc432ba58190a1994187b9" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 35415 } }
I20260806 22:50:09.551470 2906 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5d55b3dfb4fc432ba58190a1994187b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:50:09.551507 2906 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5d55b3dfb4fc432ba58190a1994187b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d55b3dfb4fc432ba58190a1994187b9, State: Initialized, Role: FOLLOWER
I20260806 22:50:09.551647 2906 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5d55b3dfb4fc432ba58190a1994187b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d55b3dfb4fc432ba58190a1994187b9" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 35415 } }
I20260806 22:50:09.551733 2906 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5d55b3dfb4fc432ba58190a1994187b9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:50:09.551761 2906 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5d55b3dfb4fc432ba58190a1994187b9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:50:09.551802 2906 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5d55b3dfb4fc432ba58190a1994187b9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:50:09.552428 2906 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5d55b3dfb4fc432ba58190a1994187b9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d55b3dfb4fc432ba58190a1994187b9" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 35415 } }
I20260806 22:50:09.552537 2906 leader_election.cc:302] T 00000000000000000000000000000000 P 5d55b3dfb4fc432ba58190a1994187b9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5d55b3dfb4fc432ba58190a1994187b9; no voters:
I20260806 22:50:09.552691 2906 leader_election.cc:288] T 00000000000000000000000000000000 P 5d55b3dfb4fc432ba58190a1994187b9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:50:09.552729 2911 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5d55b3dfb4fc432ba58190a1994187b9 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:50:09.552872 2911 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5d55b3dfb4fc432ba58190a1994187b9 [term 1 LEADER]: Becoming Leader. State: Replica: 5d55b3dfb4fc432ba58190a1994187b9, State: Running, Role: LEADER
I20260806 22:50:09.552903 2906 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5d55b3dfb4fc432ba58190a1994187b9 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:50:09.552956 2911 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5d55b3dfb4fc432ba58190a1994187b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d55b3dfb4fc432ba58190a1994187b9" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 35415 } }
I20260806 22:50:09.553273 2912 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d55b3dfb4fc432ba58190a1994187b9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5d55b3dfb4fc432ba58190a1994187b9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d55b3dfb4fc432ba58190a1994187b9" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 35415 } } }
I20260806 22:50:09.553369 2912 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d55b3dfb4fc432ba58190a1994187b9 [sys.catalog]: This master's current role is: LEADER
I20260806 22:50:09.553596 2913 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d55b3dfb4fc432ba58190a1994187b9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5d55b3dfb4fc432ba58190a1994187b9. Latest consensus state: current_term: 1 leader_uuid: "5d55b3dfb4fc432ba58190a1994187b9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d55b3dfb4fc432ba58190a1994187b9" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 35415 } } }
I20260806 22:50:09.553656 2913 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d55b3dfb4fc432ba58190a1994187b9 [sys.catalog]: This master's current role is: LEADER
W20260806 22:50:09.554579 2928 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5d55b3dfb4fc432ba58190a1994187b9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 22:50:09.554638 2928 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 22:50:09.555450 2919 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:50:09.555621 2919 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:50:09.556387 2910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:09.549986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43490 (local address 127.28.218.254:35415)
0806 22:50:09.550199 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:09.550208 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:09.550241 (+ 33us) server_negotiation.cc:407] Connection header received
0806 22:50:09.550831 (+ 590us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:09.550839 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:09.550913 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:09.551072 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:09.551318 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:09.552980 (+ 1662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:09.554193 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:09.554787 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:09.554911 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:09.555637 (+ 726us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:50:09.555641 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:50:09.555641 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:50:09.555889 (+ 248us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:50:09.555929 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:50:09.555939 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:50:09.556023 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:09.556100 (+ 77us) server_negotiation.cc:299] Negotiation successful
0806 22:50:09.556180 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":87,"threads_started":1}
I20260806 22:50:09.557044 2919 catalog_manager.cc:1347] Generated new cluster ID: 7378b4aaf1b844fea1cff1a68b35cb2a
I20260806 22:50:09.557122 2919 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:50:09.563428 29547 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:50:09.566154 2919 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:50:09.566610 2919 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:50:09.576409 2919 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5d55b3dfb4fc432ba58190a1994187b9: Generated new TSK 0
I20260806 22:50:09.576581 2919 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:50:10.445943 2910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:09.587663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43504 (local address 127.28.218.254:35415)
0806 22:50:09.587702 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:09.587704 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:09.587805 (+ 101us) server_negotiation.cc:407] Connection header received
0806 22:50:09.587879 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:09.587879 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:09.587889 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:09.587949 (+ 60us) server_negotiation.cc:226] Negotiated authn=JWT
0806 22:50:09.588109 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:09.588614 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:09.589744 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:09.589930 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:09.589977 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:09.590189 (+ 212us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 22:50:10.445700 (+855511us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0806 22:50:10.445882 (+ 182us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:43504: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":17}
W20260806 22:50:10.446009 2910 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:43504: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260806 22:50:10.446111 2938 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:50:09.587610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:35415 (local address 127.0.0.1:43504)
0806 22:50:09.587745 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:09.587755 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:09.587813 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:09.587981 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:09.587982 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:09.587984 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0806 22:50:09.588047 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:09.588050 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:09.588697 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:09.588698 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:09.589654 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:09.589657 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:09.590007 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:09.590008 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:09.590086 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:09.590091 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 22:50:10.445849 (+855758us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0806 22:50:10.445974 (+ 125us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.28.218.254:35415: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":77,"threads_started":1}
I20260806 22:50:10.446532 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 2835
2026-08-06T22:50:10Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (1038 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (88 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (89 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056586788339-29547-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:50:10 dist-test-slave-1xrh krb5kdc[2952](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:50:10 dist-test-slave-1xrh krb5kdc[2952](info): set up 2 sockets
Aug 06 22:50:10 dist-test-slave-1xrh krb5kdc[2952](info): commencing operation
krb5kdc: starting...
W20260806 22:50:11.694815 29547 mini_kdc.cc:121] Time spent starting KDC: real 1.051s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056586788339-29547-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056586788339-29547-0/krb5kdc/test-user.keytab.
Aug 06 22:50:11 dist-test-slave-1xrh krb5kdc[2952](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056611, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:50:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:50:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.218.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.218.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.218.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab.
Entry for principal kudu/127.28.218.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab.
I20260806 22:50:11.811980 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056586788339-29547-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056586788339-29547-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056586788339-29547-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.218.254:34641
--webserver_interface=127.28.218.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab
--principal=kudu/127.28.218.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.218.212:43141
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.218.254:34641
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:50:11.893635 2967 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:50:11.893838 2967 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:50:11.893860 2967 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:50:11.895045 2967 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:50:11.895087 2967 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:50:11.895102 2967 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:50:11.895114 2967 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:50:11.895143 2967 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:50:11.896675 2967 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:43141
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056586788339-29547-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056586788339-29547-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.218.254:34641
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab
--principal=kudu/127.28.218.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.218.254:34641
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.218.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2967
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056586788339-29547-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:50:11.896921 2967 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:50:11.897145 2967 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:50:11.901909 2976 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:11.901943 2974 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:11.902000 2973 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:50:11.902042 2967 server_base.cc:1034] running on GCE node
I20260806 22:50:11.902318 2967 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:50:11.902602 2967 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:50:11.903738 2967 hybrid_clock.cc:648] HybridClock initialized: now 1786056611903728 us; error 28 us; skew 500 ppm
Aug 06 22:50:11 dist-test-slave-1xrh krb5kdc[2952](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056611, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:50:11.906219 2967 init.cc:377] Logged in from keytab as kudu/127.28.218.254@KRBTEST.COM (short username kudu)
I20260806 22:50:11.906672 2967 webserver.cc:460] Webserver started at http://127.28.218.254:46517/ using document root <none> and password file <none>
I20260806 22:50:11.906903 2967 fs_manager.cc:359] Metadata directory not provided
I20260806 22:50:11.906947 2967 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:50:11.907073 2967 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:50:11.908169 2967 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056586788339-29547-0/minicluster-data/master-0/data/instance:
uuid: "f6035934843a43faa42ed0d544089afc"
format_stamp: "Formatted at 2026-08-06 22:50:11 on dist-test-slave-1xrh"
server_key: "5e55783c21023d9a1a7729ca0ceec8c3"
server_key_iv: "d1840a13cf3c416045166adb00b220ee"
server_key_version: "encryptionkey@0"
I20260806 22:50:11.908519 2967 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056586788339-29547-0/minicluster-data/master-0/wal/instance:
uuid: "f6035934843a43faa42ed0d544089afc"
format_stamp: "Formatted at 2026-08-06 22:50:11 on dist-test-slave-1xrh"
server_key: "5e55783c21023d9a1a7729ca0ceec8c3"
server_key_iv: "d1840a13cf3c416045166adb00b220ee"
server_key_version: "encryptionkey@0"
I20260806 22:50:11.909830 2967 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 22:50:11.910537 2983 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:11.910734 2967 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:50:11.910799 2967 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056586788339-29547-0/minicluster-data/master-0/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056586788339-29547-0/minicluster-data/master-0/wal
uuid: "f6035934843a43faa42ed0d544089afc"
format_stamp: "Formatted at 2026-08-06 22:50:11 on dist-test-slave-1xrh"
server_key: "5e55783c21023d9a1a7729ca0ceec8c3"
server_key_iv: "d1840a13cf3c416045166adb00b220ee"
server_key_version: "encryptionkey@0"
I20260806 22:50:11.910872 2967 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056586788339-29547-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056586788339-29547-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056586788339-29547-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:50:11.936446 2967 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:50:11.938374 2967 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:50:11.938529 2967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:50:11.942682 2967 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.254:34641
I20260806 22:50:11.942739 3040 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.254:34641 every 8 connection(s)
I20260806 22:50:11.943090 2967 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
I20260806 22:50:11.943994 3041 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:50:11.946336 3041 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6035934843a43faa42ed0d544089afc: Bootstrap starting.
I20260806 22:50:11.946569 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 2967
I20260806 22:50:11.946672 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056586788339-29547-0/minicluster-data/master-0/wal/instance
I20260806 22:50:11.946875 29547 external_mini_cluster.cc:1428] Setting key 747f52160b2817b0305d03e026c4e2e9
I20260806 22:50:11.947085 3041 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f6035934843a43faa42ed0d544089afc: Neither blocks nor log segments found. Creating new log.
I20260806 22:50:11.947489 3041 log.cc:824] T 00000000000000000000000000000000 P f6035934843a43faa42ed0d544089afc: Log is configured to *not* fsync() on all Append() calls
I20260806 22:50:11.948287 3041 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6035934843a43faa42ed0d544089afc: No bootstrap required, opened a new log
I20260806 22:50:11.950109 3041 raft_consensus.cc:348] T 00000000000000000000000000000000 P f6035934843a43faa42ed0d544089afc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6035934843a43faa42ed0d544089afc" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 34641 } }
I20260806 22:50:11.950263 3041 raft_consensus.cc:374] T 00000000000000000000000000000000 P f6035934843a43faa42ed0d544089afc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:50:11.950312 3041 raft_consensus.cc:729] T 00000000000000000000000000000000 P f6035934843a43faa42ed0d544089afc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6035934843a43faa42ed0d544089afc, State: Initialized, Role: FOLLOWER
I20260806 22:50:11.950451 3041 consensus_queue.cc:260] T 00000000000000000000000000000000 P f6035934843a43faa42ed0d544089afc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6035934843a43faa42ed0d544089afc" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 34641 } }
I20260806 22:50:11.950542 3041 raft_consensus.cc:388] T 00000000000000000000000000000000 P f6035934843a43faa42ed0d544089afc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:50:11.950587 3041 raft_consensus.cc:482] T 00000000000000000000000000000000 P f6035934843a43faa42ed0d544089afc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:50:11.950634 3041 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f6035934843a43faa42ed0d544089afc [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:50:11.951205 3041 raft_consensus.cc:504] T 00000000000000000000000000000000 P f6035934843a43faa42ed0d544089afc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6035934843a43faa42ed0d544089afc" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 34641 } }
I20260806 22:50:11.951320 3041 leader_election.cc:302] T 00000000000000000000000000000000 P f6035934843a43faa42ed0d544089afc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f6035934843a43faa42ed0d544089afc; no voters:
I20260806 22:50:11.951503 3041 leader_election.cc:288] T 00000000000000000000000000000000 P f6035934843a43faa42ed0d544089afc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:50:11.951537 3046 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f6035934843a43faa42ed0d544089afc [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:50:11.951658 3046 raft_consensus.cc:686] T 00000000000000000000000000000000 P f6035934843a43faa42ed0d544089afc [term 1 LEADER]: Becoming Leader. State: Replica: f6035934843a43faa42ed0d544089afc, State: Running, Role: LEADER
I20260806 22:50:11.951745 3046 consensus_queue.cc:237] T 00000000000000000000000000000000 P f6035934843a43faa42ed0d544089afc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6035934843a43faa42ed0d544089afc" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 34641 } }
I20260806 22:50:11.952050 3041 sys_catalog.cc:564] T 00000000000000000000000000000000 P f6035934843a43faa42ed0d544089afc [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:50:11.952214 3047 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6035934843a43faa42ed0d544089afc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f6035934843a43faa42ed0d544089afc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6035934843a43faa42ed0d544089afc" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 34641 } } }
I20260806 22:50:11.952221 3048 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6035934843a43faa42ed0d544089afc [sys.catalog]: SysCatalogTable state changed. Reason: New leader f6035934843a43faa42ed0d544089afc. Latest consensus state: current_term: 1 leader_uuid: "f6035934843a43faa42ed0d544089afc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6035934843a43faa42ed0d544089afc" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 34641 } } }
I20260806 22:50:11.952309 3047 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6035934843a43faa42ed0d544089afc [sys.catalog]: This master's current role is: LEADER
I20260806 22:50:11.952333 3048 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6035934843a43faa42ed0d544089afc [sys.catalog]: This master's current role is: LEADER
I20260806 22:50:11.952605 3050 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:50:11.953187 3050 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 06 22:50:11 dist-test-slave-1xrh krb5kdc[2952](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056611, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.218.254@KRBTEST.COM
I20260806 22:50:11.955493 3050 catalog_manager.cc:1347] Generated new cluster ID: b721b557e4f14dc78dca5f969ebb1f57
I20260806 22:50:11.955546 3050 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:50:11.957844 3044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:11.947440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56796 (local address 127.28.218.254:34641)
0806 22:50:11.947615 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:11.947625 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:11.947658 (+ 33us) server_negotiation.cc:407] Connection header received
0806 22:50:11.948193 (+ 535us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:11.948201 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:11.948251 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:11.948403 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:11.949039 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:11.949911 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:11.950840 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:11.951338 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:11.951444 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:11.954217 (+ 2773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:50:11.954221 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:50:11.954221 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:50:11.954233 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:50:11.956229 (+ 1996us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:11.956650 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:11.956652 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:11.956653 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:11.956752 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:11.956968 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:11.956969 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:11.956969 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:11.957240 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:50:11.957381 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:11.957523 (+ 142us) server_negotiation.cc:299] Negotiation successful
0806 22:50:11.957644 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":87,"threads_started":1}
I20260806 22:50:11.989738 3050 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:50:11.990553 3050 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:50:12.008476 3050 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f6035934843a43faa42ed0d544089afc: Generated new TSK 0
I20260806 22:50:12.008700 3050 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.28.218.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.218.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.218.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.193.keytab.
Entry for principal kudu/127.28.218.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.193.keytab.
I20260806 22:50:12.045476 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056586788339-29547-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056586788339-29547-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056586788339-29547-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056586788339-29547-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.218.193:0
--local_ip_for_outbound_sockets=127.28.218.193
--webserver_interface=127.28.218.193
--webserver_port=0
--tserver_master_addrs=127.28.218.254:34641
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.193.keytab
--principal=kudu/127.28.218.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.218.212:43141
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:50:12.129055 3069 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:50:12.129244 3069 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:50:12.129267 3069 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:50:12.130430 3069 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:50:12.130473 3069 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:50:12.130499 3069 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.218.193
I20260806 22:50:12.132032 3069 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:43141
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056586788339-29547-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056586788339-29547-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.193.keytab
--principal=kudu/127.28.218.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.218.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056586788339-29547-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.218.193
--webserver_port=0
--tserver_master_addrs=127.28.218.254:34641
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3069
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.218.193
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056586788339-29547-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:50:12.132256 3069 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:50:12.132486 3069 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:50:12.137522 3074 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:12.137501 3079 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:12.137501 3076 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:50:12.137590 3069 server_base.cc:1034] running on GCE node
I20260806 22:50:12.137936 3069 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:50:12.138156 3069 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:50:12.139288 3069 hybrid_clock.cc:648] HybridClock initialized: now 1786056612139276 us; error 31 us; skew 500 ppm
Aug 06 22:50:12 dist-test-slave-1xrh krb5kdc[2952](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056612, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:50:12.141800 3069 init.cc:377] Logged in from keytab as kudu/127.28.218.193@KRBTEST.COM (short username kudu)
I20260806 22:50:12.142220 3069 webserver.cc:460] Webserver started at http://127.28.218.193:40413/ using document root <none> and password file <none>
I20260806 22:50:12.142458 3069 fs_manager.cc:359] Metadata directory not provided
I20260806 22:50:12.142524 3069 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:50:12.142665 3069 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:50:12.143751 3069 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056586788339-29547-0/minicluster-data/ts-0/data/instance:
uuid: "1d5246762de44f79b9a29dc6bf000fdd"
format_stamp: "Formatted at 2026-08-06 22:50:12 on dist-test-slave-1xrh"
server_key: "982b19e56effa6802f146294ac9e5a7f"
server_key_iv: "f8781e259e8afe86aa44c521f4558be8"
server_key_version: "encryptionkey@0"
I20260806 22:50:12.144114 3069 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056586788339-29547-0/minicluster-data/ts-0/wal/instance:
uuid: "1d5246762de44f79b9a29dc6bf000fdd"
format_stamp: "Formatted at 2026-08-06 22:50:12 on dist-test-slave-1xrh"
server_key: "982b19e56effa6802f146294ac9e5a7f"
server_key_iv: "f8781e259e8afe86aa44c521f4558be8"
server_key_version: "encryptionkey@0"
I20260806 22:50:12.145404 3069 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:50:12.146076 3087 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:12.146237 3069 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:50:12.146334 3069 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056586788339-29547-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056586788339-29547-0/minicluster-data/ts-0/wal
uuid: "1d5246762de44f79b9a29dc6bf000fdd"
format_stamp: "Formatted at 2026-08-06 22:50:12 on dist-test-slave-1xrh"
server_key: "982b19e56effa6802f146294ac9e5a7f"
server_key_iv: "f8781e259e8afe86aa44c521f4558be8"
server_key_version: "encryptionkey@0"
I20260806 22:50:12.146409 3069 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056586788339-29547-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056586788339-29547-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056586788339-29547-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:50:12.155340 3069 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:50:12.157438 3069 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:50:12.157586 3069 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:50:12.157793 3069 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:50:12.158071 3069 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:50:12.158120 3069 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:12.158154 3069 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:50:12.158178 3069 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:12.163699 3069 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.193:34797
I20260806 22:50:12.163757 3203 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.193:34797 every 8 connection(s)
I20260806 22:50:12.164055 3069 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056586788339-29547-0/minicluster-data/ts-0/data/info.pb
Aug 06 22:50:12 dist-test-slave-1xrh krb5kdc[2952](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056612, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.193@KRBTEST.COM for kudu/127.28.218.254@KRBTEST.COM
I20260806 22:50:12.170316 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 3069
I20260806 22:50:12.170470 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056586788339-29547-0/minicluster-data/ts-0/wal/instance
I20260806 22:50:12.170668 29547 external_mini_cluster.cc:1428] Setting key b20133cf44d58caa053e48be86b47055
I20260806 22:50:12.173094 3044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:12.164738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.193:58707 (local address 127.28.218.254:34641)
0806 22:50:12.164796 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:12.164798 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:12.164958 (+ 160us) server_negotiation.cc:407] Connection header received
0806 22:50:12.165847 (+ 889us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:12.165848 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:12.165857 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:12.165898 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:12.166438 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:12.166918 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:12.167898 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:12.168097 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:12.168143 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:12.170038 (+ 1895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:50:12.170039 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:50:12.170040 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:50:12.170045 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:50:12.171684 (+ 1639us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:12.172064 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:12.172065 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:12.172066 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:12.172125 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:12.172444 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:12.172446 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:12.172446 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:12.172561 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:50:12.172634 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:12.172894 (+ 260us) server_negotiation.cc:299] Negotiation successful
0806 22:50:12.172998 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 22:50:12.173210 3206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:12.164699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:34641 (local address 127.28.218.193:58707)
0806 22:50:12.164842 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:12.164881 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:12.165748 (+ 867us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:12.165958 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:12.165961 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:12.166189 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:12.166363 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:12.166367 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:12.166992 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:12.166993 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:12.167822 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:12.167825 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:12.168160 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:12.168161 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:12.168249 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:12.168704 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:50:12.168712 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:12.169960 (+ 1248us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:50:12.171762 (+ 1802us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:12.171768 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:12.171771 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:12.171989 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:12.172194 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:12.172194 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:12.172195 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:12.172340 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:12.172625 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:50:12.172632 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:50:12.172754 (+ 122us) client_negotiation.cc:769] Sending connection context
0806 22:50:12.172850 (+ 96us) client_negotiation.cc:240] Negotiation successful
0806 22:50:12.173001 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":79,"threads_started":1}
I20260806 22:50:12.173671 3204 heartbeater.cc:347] Connected to a master server at 127.28.218.254:34641
I20260806 22:50:12.173777 3204 heartbeater.cc:464] Registering TS with master...
I20260806 22:50:12.173939 3204 heartbeater.cc:511] Master 127.28.218.254:34641 requested a full tablet report, sending...
I20260806 22:50:12.174450 3003 ts_manager.cc:194] Registered new tserver with Master: 1d5246762de44f79b9a29dc6bf000fdd (127.28.218.193:34797)
I20260806 22:50:12.175025 3003 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.218.193@KRBTEST.COM'} at 127.28.218.193:58707
WARNING: no policy specified for kudu/127.28.218.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.218.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.218.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.194.keytab.
Entry for principal kudu/127.28.218.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.194.keytab.
I20260806 22:50:12.197772 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056586788339-29547-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056586788339-29547-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056586788339-29547-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056586788339-29547-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.218.194:0
--local_ip_for_outbound_sockets=127.28.218.194
--webserver_interface=127.28.218.194
--webserver_port=0
--tserver_master_addrs=127.28.218.254:34641
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.194.keytab
--principal=kudu/127.28.218.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.218.212:43141
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:50:12.277783 3210 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:50:12.277976 3210 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:50:12.277998 3210 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:50:12.279090 3210 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:50:12.279165 3210 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:50:12.279206 3210 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.218.194
I20260806 22:50:12.280587 3210 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:43141
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056586788339-29547-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056586788339-29547-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.194.keytab
--principal=kudu/127.28.218.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.218.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056586788339-29547-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.218.194
--webserver_port=0
--tserver_master_addrs=127.28.218.254:34641
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3210
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.218.194
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056586788339-29547-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:50:12.280756 3210 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:50:12.280925 3210 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:50:12.285780 3216 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:12.285810 3215 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:12.285776 3218 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:50:12.285826 3210 server_base.cc:1034] running on GCE node
I20260806 22:50:12.286233 3210 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:50:12.286474 3210 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:50:12.287658 3210 hybrid_clock.cc:648] HybridClock initialized: now 1786056612287643 us; error 48 us; skew 500 ppm
Aug 06 22:50:12 dist-test-slave-1xrh krb5kdc[2952](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056612, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:50:12.290385 3210 init.cc:377] Logged in from keytab as kudu/127.28.218.194@KRBTEST.COM (short username kudu)
I20260806 22:50:12.290827 3210 webserver.cc:460] Webserver started at http://127.28.218.194:45901/ using document root <none> and password file <none>
I20260806 22:50:12.291062 3210 fs_manager.cc:359] Metadata directory not provided
I20260806 22:50:12.291152 3210 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:50:12.291312 3210 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:50:12.292517 3210 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056586788339-29547-0/minicluster-data/ts-1/data/instance:
uuid: "2ee060b873294a57a754a04499910ffc"
format_stamp: "Formatted at 2026-08-06 22:50:12 on dist-test-slave-1xrh"
server_key: "a8db8dd8dfd935a6c5b54a46543a1987"
server_key_iv: "2a4de2690551bb2b5b50f06fc3683aa9"
server_key_version: "encryptionkey@0"
I20260806 22:50:12.292898 3210 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056586788339-29547-0/minicluster-data/ts-1/wal/instance:
uuid: "2ee060b873294a57a754a04499910ffc"
format_stamp: "Formatted at 2026-08-06 22:50:12 on dist-test-slave-1xrh"
server_key: "a8db8dd8dfd935a6c5b54a46543a1987"
server_key_iv: "2a4de2690551bb2b5b50f06fc3683aa9"
server_key_version: "encryptionkey@0"
I20260806 22:50:12.294288 3210 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:50:12.295047 3228 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:12.295233 3210 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:50:12.295332 3210 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056586788339-29547-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056586788339-29547-0/minicluster-data/ts-1/wal
uuid: "2ee060b873294a57a754a04499910ffc"
format_stamp: "Formatted at 2026-08-06 22:50:12 on dist-test-slave-1xrh"
server_key: "a8db8dd8dfd935a6c5b54a46543a1987"
server_key_iv: "2a4de2690551bb2b5b50f06fc3683aa9"
server_key_version: "encryptionkey@0"
I20260806 22:50:12.295423 3210 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056586788339-29547-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056586788339-29547-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056586788339-29547-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:50:12.316823 3210 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:50:12.318840 3210 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:50:12.319039 3210 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:50:12.319309 3210 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:50:12.319648 3210 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:50:12.319705 3210 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:12.319761 3210 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:50:12.319790 3210 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:12.325963 3210 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.194:46693
I20260806 22:50:12.326018 3347 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.194:46693 every 8 connection(s)
I20260806 22:50:12.326354 3210 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056586788339-29547-0/minicluster-data/ts-1/data/info.pb
I20260806 22:50:12.332887 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 3210
I20260806 22:50:12.333024 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056586788339-29547-0/minicluster-data/ts-1/wal/instance
Aug 06 22:50:12 dist-test-slave-1xrh krb5kdc[2952](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056612, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.194@KRBTEST.COM for kudu/127.28.218.254@KRBTEST.COM
I20260806 22:50:12.333287 29547 external_mini_cluster.cc:1428] Setting key 82f1a7f2f5f31f8cef9f606c7e1033ad
I20260806 22:50:12.336766 3044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:12.327171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.194:53823 (local address 127.28.218.254:34641)
0806 22:50:12.327232 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:12.327235 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:12.327456 (+ 221us) server_negotiation.cc:407] Connection header received
0806 22:50:12.328514 (+ 1058us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:12.328515 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:12.328528 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:12.328599 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:12.329277 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:12.329832 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:12.330902 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:12.331194 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:12.331271 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:12.333666 (+ 2395us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:50:12.333667 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:50:12.333667 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:50:12.333674 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:50:12.335367 (+ 1693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:12.335750 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:12.335751 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:12.335752 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:12.335816 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:12.336094 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:12.336096 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:12.336096 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:12.336252 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:50:12.336307 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:12.336637 (+ 330us) server_negotiation.cc:299] Negotiation successful
0806 22:50:12.336702 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260806 22:50:12.337065 3350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:12.327110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:34641 (local address 127.28.218.194:53823)
0806 22:50:12.327314 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:12.327361 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:12.328395 (+ 1034us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:12.328674 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:12.328678 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:12.328959 (+ 281us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:12.329161 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:12.329165 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:12.329937 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:12.329938 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:12.330805 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:12.330808 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:12.331291 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:12.331292 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:12.331425 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:12.331920 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:50:12.331928 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:12.333555 (+ 1627us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:50:12.335443 (+ 1888us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:12.335450 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:12.335452 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:12.335678 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:12.335879 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:12.335880 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:12.335880 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:12.336017 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:12.336317 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:50:12.336323 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:50:12.336495 (+ 172us) client_negotiation.cc:769] Sending connection context
0806 22:50:12.336619 (+ 124us) client_negotiation.cc:240] Negotiation successful
0806 22:50:12.336797 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":107,"threads_started":1}
I20260806 22:50:12.337615 3348 heartbeater.cc:347] Connected to a master server at 127.28.218.254:34641
I20260806 22:50:12.337728 3348 heartbeater.cc:464] Registering TS with master...
I20260806 22:50:12.337927 3348 heartbeater.cc:511] Master 127.28.218.254:34641 requested a full tablet report, sending...
I20260806 22:50:12.338285 3003 ts_manager.cc:194] Registered new tserver with Master: 2ee060b873294a57a754a04499910ffc (127.28.218.194:46693)
I20260806 22:50:12.338689 3003 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.218.194@KRBTEST.COM'} at 127.28.218.194:53823
WARNING: no policy specified for kudu/127.28.218.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.218.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.218.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.195.keytab.
Entry for principal kudu/127.28.218.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.195.keytab.
I20260806 22:50:12.361483 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056586788339-29547-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056586788339-29547-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056586788339-29547-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056586788339-29547-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.218.195:0
--local_ip_for_outbound_sockets=127.28.218.195
--webserver_interface=127.28.218.195
--webserver_port=0
--tserver_master_addrs=127.28.218.254:34641
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.195.keytab
--principal=kudu/127.28.218.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.218.212:43141
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:50:12.442871 3353 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:50:12.443069 3353 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:50:12.443091 3353 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:50:12.444232 3353 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:50:12.444275 3353 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:50:12.444322 3353 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.218.195
I20260806 22:50:12.445756 3353 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:43141
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056586788339-29547-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056586788339-29547-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.195.keytab
--principal=kudu/127.28.218.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.218.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056586788339-29547-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.218.195
--webserver_port=0
--tserver_master_addrs=127.28.218.254:34641
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3353
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.218.195
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056586788339-29547-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:50:12.446004 3353 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:50:12.446214 3353 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:50:12.451169 3358 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:50:12.451210 3353 server_base.cc:1034] running on GCE node
W20260806 22:50:12.451253 3359 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:12.451263 3361 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:50:12.451642 3353 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:50:12.451879 3353 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:50:12.453075 3353 hybrid_clock.cc:648] HybridClock initialized: now 1786056612453056 us; error 34 us; skew 500 ppm
Aug 06 22:50:12 dist-test-slave-1xrh krb5kdc[2952](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056612, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:50:12.455595 3353 init.cc:377] Logged in from keytab as kudu/127.28.218.195@KRBTEST.COM (short username kudu)
I20260806 22:50:12.456048 3353 webserver.cc:460] Webserver started at http://127.28.218.195:39509/ using document root <none> and password file <none>
I20260806 22:50:12.456274 3353 fs_manager.cc:359] Metadata directory not provided
I20260806 22:50:12.456337 3353 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:50:12.456486 3353 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:50:12.457544 3353 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056586788339-29547-0/minicluster-data/ts-2/data/instance:
uuid: "f7478fe03b7c4678afb87c7772be52a9"
format_stamp: "Formatted at 2026-08-06 22:50:12 on dist-test-slave-1xrh"
server_key: "14e5259e776aca92ab1043bad00ba0f3"
server_key_iv: "dfe7cd359d7bd5d714bcacf9514ece4b"
server_key_version: "encryptionkey@0"
I20260806 22:50:12.457887 3353 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056586788339-29547-0/minicluster-data/ts-2/wal/instance:
uuid: "f7478fe03b7c4678afb87c7772be52a9"
format_stamp: "Formatted at 2026-08-06 22:50:12 on dist-test-slave-1xrh"
server_key: "14e5259e776aca92ab1043bad00ba0f3"
server_key_iv: "dfe7cd359d7bd5d714bcacf9514ece4b"
server_key_version: "encryptionkey@0"
I20260806 22:50:12.459240 3353 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 22:50:12.459954 3370 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:12.460117 3353 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:50:12.460210 3353 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056586788339-29547-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056586788339-29547-0/minicluster-data/ts-2/wal
uuid: "f7478fe03b7c4678afb87c7772be52a9"
format_stamp: "Formatted at 2026-08-06 22:50:12 on dist-test-slave-1xrh"
server_key: "14e5259e776aca92ab1043bad00ba0f3"
server_key_iv: "dfe7cd359d7bd5d714bcacf9514ece4b"
server_key_version: "encryptionkey@0"
I20260806 22:50:12.460287 3353 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056586788339-29547-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056586788339-29547-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056586788339-29547-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:50:12.468626 3353 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:50:12.470260 3353 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:50:12.470407 3353 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:50:12.470608 3353 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:50:12.470901 3353 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:50:12.470956 3353 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:12.471004 3353 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:50:12.471031 3353 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:12.477080 3353 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.195:37965
I20260806 22:50:12.477123 3491 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.195:37965 every 8 connection(s)
I20260806 22:50:12.477406 3353 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056586788339-29547-0/minicluster-data/ts-2/data/info.pb
Aug 06 22:50:12 dist-test-slave-1xrh krb5kdc[2952](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056612, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.195@KRBTEST.COM for kudu/127.28.218.254@KRBTEST.COM
I20260806 22:50:12.485755 3044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:12.478073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.195:52761 (local address 127.28.218.254:34641)
0806 22:50:12.478132 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:12.478134 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:12.478296 (+ 162us) server_negotiation.cc:407] Connection header received
0806 22:50:12.479228 (+ 932us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:12.479230 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:12.479239 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:12.479292 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:12.479806 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:12.480309 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:12.481219 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:12.481396 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:12.481471 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:12.483459 (+ 1988us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:50:12.483460 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:50:12.483461 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:50:12.483466 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:50:12.484691 (+ 1225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:12.485042 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:12.485043 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:12.485043 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:12.485092 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:12.485281 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:12.485281 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:12.485281 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:12.485376 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:50:12.485426 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:12.485634 (+ 208us) server_negotiation.cc:299] Negotiation successful
0806 22:50:12.485694 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 22:50:12.485926 3494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:12.478035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:34641 (local address 127.28.218.195:52761)
0806 22:50:12.478178 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:12.478217 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:12.479086 (+ 869us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:12.479341 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:12.479344 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:12.479570 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:12.479742 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:12.479745 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:12.480380 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:12.480381 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:12.481150 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:12.481152 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:12.481488 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:12.481488 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:12.481579 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:12.482011 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:50:12.482018 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:12.483372 (+ 1354us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:50:12.484759 (+ 1387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:12.484764 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:12.484765 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:12.484981 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:12.485145 (+ 164us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:12.485145 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:12.485145 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:12.485232 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:12.485424 (+ 192us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:50:12.485428 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:50:12.485530 (+ 102us) client_negotiation.cc:769] Sending connection context
0806 22:50:12.485608 (+ 78us) client_negotiation.cc:240] Negotiation successful
0806 22:50:12.485715 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":78,"threads_started":1}
I20260806 22:50:12.486397 3492 heartbeater.cc:347] Connected to a master server at 127.28.218.254:34641
I20260806 22:50:12.486511 3492 heartbeater.cc:464] Registering TS with master...
I20260806 22:50:12.486577 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 3353
I20260806 22:50:12.486671 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056586788339-29547-0/minicluster-data/ts-2/wal/instance
I20260806 22:50:12.486708 3492 heartbeater.cc:511] Master 127.28.218.254:34641 requested a full tablet report, sending...
I20260806 22:50:12.486860 29547 external_mini_cluster.cc:1428] Setting key 3ecf0fb45d40e0b8813a6990fa218ad9
I20260806 22:50:12.487099 3003 ts_manager.cc:194] Registered new tserver with Master: f7478fe03b7c4678afb87c7772be52a9 (127.28.218.195:37965)
I20260806 22:50:12.487551 3003 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.218.195@KRBTEST.COM'} at 127.28.218.195:52761
I20260806 22:50:12.487717 29547 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056586788339-29547-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056586788339-29547-0/krb5kdc/test-admin.keytab.
Aug 06 22:50:12 dist-test-slave-1xrh krb5kdc[2952](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056612, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:50:12.503988 29547 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260806 22:50:12.505558 3499 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:50:12.504760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46052 (local address 127.28.218.193:34797)
0806 22:50:12.504920 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:12.504933 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:12.504944 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:50:12.504968 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:12.504970 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:12.504983 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:12.505052 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:12.505451 (+ 399us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46052: BlockingRecv error: recv got EOF from 127.0.0.1:46052 (error 108)
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":93,"threads_started":1}
W20260806 22:50:12.505726 29547 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056586788339-29547-0/krb5kdc/krb5cc' of type FILE
Aug 06 22:50:12 dist-test-slave-1xrh krb5kdc[2952](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056612, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:50:12.506671 29547 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 22:50:12 dist-test-slave-1xrh krb5kdc[2952](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056612, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:50:12.507560 29547 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 22:50:12 dist-test-slave-1xrh krb5kdc[2952](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056612, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.218.193@KRBTEST.COM
I20260806 22:50:12.515239 3499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:12.508022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46060 (local address 127.28.218.193:34797)
0806 22:50:12.508082 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:12.508085 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:12.508142 (+ 57us) server_negotiation.cc:407] Connection header received
0806 22:50:12.508238 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:12.508239 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:12.508250 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:12.508299 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:12.508731 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:12.509448 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:12.510156 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:12.510473 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:12.510549 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:12.512635 (+ 2086us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:50:12.512643 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:50:12.512644 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:50:12.512651 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:50:12.514104 (+ 1453us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:12.514424 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:12.514428 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:12.514428 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:12.514506 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:12.514687 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:12.514688 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:12.514689 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:12.514822 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:50:12.514999 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:12.515041 (+ 42us) server_negotiation.cc:299] Negotiation successful
0806 22:50:12.515116 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
W20260806 22:50:12.516907 3499 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:50:12.516361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46066 (local address 127.28.218.193:34797)
0806 22:50:12.516419 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:12.516421 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:12.516477 (+ 56us) server_negotiation.cc:407] Connection header received
0806 22:50:12.516546 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:12.516546 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:12.516557 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:12.516604 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:12.516861 (+ 257us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46066: BlockingRecv error: recv got EOF from 127.0.0.1:46066 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
W20260806 22:50:12.516971 29547 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056586788339-29547-0/krb5kdc/krb5cc' of type FILE
Aug 06 22:50:12 dist-test-slave-1xrh krb5kdc[2952](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056612, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:50:12.517812 29547 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 22:50:12 dist-test-slave-1xrh krb5kdc[2952](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056612, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:50:12.518573 29547 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 22:50:12 dist-test-slave-1xrh krb5kdc[2952](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056612, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.218.193@KRBTEST.COM
I20260806 22:50:12.524883 3499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:12.519044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46068 (local address 127.28.218.193:34797)
0806 22:50:12.519096 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:12.519098 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:12.519147 (+ 49us) server_negotiation.cc:407] Connection header received
0806 22:50:12.519229 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:12.519230 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:12.519240 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:12.519295 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:12.519652 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:12.520083 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:12.520715 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:12.520903 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:12.520977 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:12.522706 (+ 1729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:50:12.522707 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:50:12.522707 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:50:12.522714 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:50:12.523839 (+ 1125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:12.524201 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:12.524203 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:12.524203 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:12.524253 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:12.524447 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:12.524447 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:12.524448 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:12.524536 (+ 88us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:50:12.524620 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:12.524735 (+ 115us) server_negotiation.cc:299] Negotiation successful
0806 22:50:12.524801 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
W20260806 22:50:12.526235 3499 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:50:12.525726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46076 (local address 127.28.218.193:34797)
0806 22:50:12.525781 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:12.525783 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:12.525819 (+ 36us) server_negotiation.cc:407] Connection header received
0806 22:50:12.525887 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:12.525888 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:12.525896 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:12.525943 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:12.526194 (+ 251us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46076: BlockingRecv error: recv got EOF from 127.0.0.1:46076 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
W20260806 22:50:12.526311 29547 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056586788339-29547-0/krb5kdc/krb5cc' of type FILE
Aug 06 22:50:12 dist-test-slave-1xrh krb5kdc[2952](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056612, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:50:12.527428 29547 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 22:50:12 dist-test-slave-1xrh krb5kdc[2952](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056612, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:50:12.528388 29547 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 22:50:12 dist-test-slave-1xrh krb5kdc[2952](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056612, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.218.193@KRBTEST.COM
I20260806 22:50:12.534494 3499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:12.528840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46084 (local address 127.28.218.193:34797)
0806 22:50:12.528891 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:12.528892 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:12.528921 (+ 29us) server_negotiation.cc:407] Connection header received
0806 22:50:12.528966 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:12.528967 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:12.528974 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:12.529016 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:12.529312 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:12.529743 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:12.530310 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:12.530471 (+ 161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:12.530515 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:12.532335 (+ 1820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:50:12.532336 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:50:12.532336 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:50:12.532342 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:50:12.533449 (+ 1107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:12.533766 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:12.533766 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:12.533767 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:12.533816 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:12.534044 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:12.534045 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:12.534045 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:12.534161 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:50:12.534230 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:12.534324 (+ 94us) server_negotiation.cc:299] Negotiation successful
0806 22:50:12.534377 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 22:50:12.535225 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 3069
I20260806 22:50:12.542161 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 3210
I20260806 22:50:12.547478 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 3353
I20260806 22:50:12.552558 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 2967
2026-08-06T22:50:12Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (1929 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:50:12 dist-test-slave-1xrh krb5kdc[3512](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:50:12 dist-test-slave-1xrh krb5kdc[3512](info): set up 2 sockets
Aug 06 22:50:12 dist-test-slave-1xrh krb5kdc[3512](info): commencing operation
krb5kdc: starting...
W20260806 22:50:13.614617 29547 mini_kdc.cc:121] Time spent starting KDC: real 1.041s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/krb5kdc/test-user.keytab.
Aug 06 22:50:13 dist-test-slave-1xrh krb5kdc[3512](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056613, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:50:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:50:13Z Disabled control of system clock
WARNING: no policy specified for oryx/127.28.218.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.28.218.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.28.218.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/krb5kdc/oryx_127.28.218.254.keytab.
Entry for principal oryx/127.28.218.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/krb5kdc/oryx_127.28.218.254.keytab.
I20260806 22:50:13.733906 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.218.254:44755
--webserver_interface=127.28.218.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/krb5kdc/oryx_127.28.218.254.keytab
--principal=oryx/127.28.218.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.218.212:37527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.218.254:44755
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:50:13.826957 3525 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:50:13.827272 3525 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:50:13.827325 3525 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:50:13.828599 3525 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:50:13.828667 3525 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:50:13.828697 3525 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:50:13.828732 3525 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:50:13.828760 3525 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:50:13.830354 3525 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:37527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.218.254:44755
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/krb5kdc/oryx_127.28.218.254.keytab
--principal=oryx/127.28.218.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.218.254:44755
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.218.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3525
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:50:13.830631 3525 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:50:13.830868 3525 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:50:13.835944 3533 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:50:13.836047 3525 server_base.cc:1034] running on GCE node
W20260806 22:50:13.835947 3531 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:13.836035 3530 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:50:13.836454 3525 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:50:13.836740 3525 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:50:13.837898 3525 hybrid_clock.cc:648] HybridClock initialized: now 1786056613837890 us; error 34 us; skew 500 ppm
Aug 06 22:50:13 dist-test-slave-1xrh krb5kdc[3512](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056613, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.218.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:50:13.840440 3525 init.cc:377] Logged in from keytab as oryx/127.28.218.254@KRBTEST.COM (short username oryx)
I20260806 22:50:13.840838 3525 webserver.cc:460] Webserver started at http://127.28.218.254:45333/ using document root <none> and password file <none>
I20260806 22:50:13.841056 3525 fs_manager.cc:359] Metadata directory not provided
I20260806 22:50:13.841121 3525 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:50:13.841270 3525 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:50:13.842283 3525 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/master-0/data/instance:
uuid: "20186b499bd34a7d9d497ba785db0fc9"
format_stamp: "Formatted at 2026-08-06 22:50:13 on dist-test-slave-1xrh"
server_key: "8ebe8f0c736fd209380d618ab7b2cb23"
server_key_iv: "22b1ec6d95d91c27ab97a63298b19a5f"
server_key_version: "encryptionkey@0"
I20260806 22:50:13.842639 3525 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/master-0/wal/instance:
uuid: "20186b499bd34a7d9d497ba785db0fc9"
format_stamp: "Formatted at 2026-08-06 22:50:13 on dist-test-slave-1xrh"
server_key: "8ebe8f0c736fd209380d618ab7b2cb23"
server_key_iv: "22b1ec6d95d91c27ab97a63298b19a5f"
server_key_version: "encryptionkey@0"
I20260806 22:50:13.844060 3525 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:50:13.844753 3540 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:13.844908 3525 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:13.845023 3525 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/master-0/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/master-0/wal
uuid: "20186b499bd34a7d9d497ba785db0fc9"
format_stamp: "Formatted at 2026-08-06 22:50:13 on dist-test-slave-1xrh"
server_key: "8ebe8f0c736fd209380d618ab7b2cb23"
server_key_iv: "22b1ec6d95d91c27ab97a63298b19a5f"
server_key_version: "encryptionkey@0"
I20260806 22:50:13.845093 3525 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:50:13.863924 3525 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:50:13.865676 3525 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:50:13.865828 3525 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:50:13.869823 3525 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.254:44755
I20260806 22:50:13.869884 3597 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.254:44755 every 8 connection(s)
I20260806 22:50:13.870211 3525 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
I20260806 22:50:13.871297 3598 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:50:13.873512 3598 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 20186b499bd34a7d9d497ba785db0fc9: Bootstrap starting.
I20260806 22:50:13.874050 3598 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 20186b499bd34a7d9d497ba785db0fc9: Neither blocks nor log segments found. Creating new log.
I20260806 22:50:13.874277 3598 log.cc:824] T 00000000000000000000000000000000 P 20186b499bd34a7d9d497ba785db0fc9: Log is configured to *not* fsync() on all Append() calls
I20260806 22:50:13.874965 3598 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 20186b499bd34a7d9d497ba785db0fc9: No bootstrap required, opened a new log
I20260806 22:50:13.876607 3598 raft_consensus.cc:348] T 00000000000000000000000000000000 P 20186b499bd34a7d9d497ba785db0fc9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20186b499bd34a7d9d497ba785db0fc9" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 44755 } }
I20260806 22:50:13.876740 3598 raft_consensus.cc:374] T 00000000000000000000000000000000 P 20186b499bd34a7d9d497ba785db0fc9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:50:13.876762 3598 raft_consensus.cc:729] T 00000000000000000000000000000000 P 20186b499bd34a7d9d497ba785db0fc9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20186b499bd34a7d9d497ba785db0fc9, State: Initialized, Role: FOLLOWER
I20260806 22:50:13.876893 3598 consensus_queue.cc:260] T 00000000000000000000000000000000 P 20186b499bd34a7d9d497ba785db0fc9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20186b499bd34a7d9d497ba785db0fc9" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 44755 } }
I20260806 22:50:13.876968 3598 raft_consensus.cc:388] T 00000000000000000000000000000000 P 20186b499bd34a7d9d497ba785db0fc9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:50:13.876991 3598 raft_consensus.cc:482] T 00000000000000000000000000000000 P 20186b499bd34a7d9d497ba785db0fc9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:50:13.877018 3598 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 20186b499bd34a7d9d497ba785db0fc9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:50:13.877749 3598 raft_consensus.cc:504] T 00000000000000000000000000000000 P 20186b499bd34a7d9d497ba785db0fc9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20186b499bd34a7d9d497ba785db0fc9" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 44755 } }
I20260806 22:50:13.877840 3598 leader_election.cc:302] T 00000000000000000000000000000000 P 20186b499bd34a7d9d497ba785db0fc9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 20186b499bd34a7d9d497ba785db0fc9; no voters:
I20260806 22:50:13.877956 3598 leader_election.cc:288] T 00000000000000000000000000000000 P 20186b499bd34a7d9d497ba785db0fc9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:50:13.878019 3601 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 20186b499bd34a7d9d497ba785db0fc9 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:50:13.878208 3598 sys_catalog.cc:564] T 00000000000000000000000000000000 P 20186b499bd34a7d9d497ba785db0fc9 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:50:13.878194 3601 raft_consensus.cc:686] T 00000000000000000000000000000000 P 20186b499bd34a7d9d497ba785db0fc9 [term 1 LEADER]: Becoming Leader. State: Replica: 20186b499bd34a7d9d497ba785db0fc9, State: Running, Role: LEADER
I20260806 22:50:13.878425 3601 consensus_queue.cc:237] T 00000000000000000000000000000000 P 20186b499bd34a7d9d497ba785db0fc9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20186b499bd34a7d9d497ba785db0fc9" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 44755 } }
I20260806 22:50:13.878819 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 3525
I20260806 22:50:13.878944 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/master-0/wal/instance
I20260806 22:50:13.879169 29547 external_mini_cluster.cc:1428] Setting key a494a5265945f82312274ba09d98e109
I20260806 22:50:13.879598 3603 sys_catalog.cc:455] T 00000000000000000000000000000000 P 20186b499bd34a7d9d497ba785db0fc9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 20186b499bd34a7d9d497ba785db0fc9. Latest consensus state: current_term: 1 leader_uuid: "20186b499bd34a7d9d497ba785db0fc9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20186b499bd34a7d9d497ba785db0fc9" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 44755 } } }
I20260806 22:50:13.879992 3603 sys_catalog.cc:458] T 00000000000000000000000000000000 P 20186b499bd34a7d9d497ba785db0fc9 [sys.catalog]: This master's current role is: LEADER
I20260806 22:50:13.879750 3602 sys_catalog.cc:455] T 00000000000000000000000000000000 P 20186b499bd34a7d9d497ba785db0fc9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "20186b499bd34a7d9d497ba785db0fc9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20186b499bd34a7d9d497ba785db0fc9" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 44755 } } }
I20260806 22:50:13.880155 3602 sys_catalog.cc:458] T 00000000000000000000000000000000 P 20186b499bd34a7d9d497ba785db0fc9 [sys.catalog]: This master's current role is: LEADER
W20260806 22:50:13.880187 3616 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 20186b499bd34a7d9d497ba785db0fc9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 22:50:13.880275 3616 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 22:50:13.880357 3622 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:50:13.880537 3622 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:50:13.881889 3622 catalog_manager.cc:1347] Generated new cluster ID: 49a945fc1aed48f391333d0beb7d3f21
I20260806 22:50:13.881945 3622 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 22:50:13 dist-test-slave-1xrh krb5kdc[3512](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056613, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.28.218.254@KRBTEST.COM
I20260806 22:50:13.889688 3620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:13.879819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51786 (local address 127.28.218.254:44755)
0806 22:50:13.879960 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:13.879969 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:13.880005 (+ 36us) server_negotiation.cc:407] Connection header received
0806 22:50:13.880412 (+ 407us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:13.880419 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:13.880475 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:13.880578 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:13.881468 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:13.882492 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:13.883282 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:13.883695 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:13.883817 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:13.886041 (+ 2224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:50:13.886044 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:50:13.886045 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:50:13.886056 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:50:13.888205 (+ 2149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:13.888567 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:13.888570 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:13.888570 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:13.888668 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:13.888873 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:13.888874 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:13.888874 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:13.889109 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:50:13.889276 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:13.889344 (+ 68us) server_negotiation.cc:299] Negotiation successful
0806 22:50:13.889483 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97,"thread_start_us":75,"threads_started":1}
I20260806 22:50:13.900746 3622 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:50:13.901245 3622 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:50:13.911636 3622 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 20186b499bd34a7d9d497ba785db0fc9: Generated new TSK 0
I20260806 22:50:13.911828 3622 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.28.218.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.28.218.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.28.218.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/krb5kdc/oryx_127.28.218.193.keytab.
Entry for principal oryx/127.28.218.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/krb5kdc/oryx_127.28.218.193.keytab.
I20260806 22:50:13.942553 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.218.193:0
--local_ip_for_outbound_sockets=127.28.218.193
--webserver_interface=127.28.218.193
--webserver_port=0
--tserver_master_addrs=127.28.218.254:44755
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/krb5kdc/oryx_127.28.218.193.keytab
--principal=oryx/127.28.218.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.218.212:37527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:50:14.044550 3626 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:50:14.044740 3626 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:50:14.044763 3626 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:50:14.045862 3626 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:50:14.045903 3626 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:50:14.045929 3626 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.218.193
I20260806 22:50:14.047400 3626 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:37527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/krb5kdc/oryx_127.28.218.193.keytab
--principal=oryx/127.28.218.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.218.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.218.193
--webserver_port=0
--tserver_master_addrs=127.28.218.254:44755
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3626
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.218.193
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:50:14.047680 3626 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:50:14.047943 3626 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:50:14.052816 3634 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:14.052851 3632 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:14.052829 3631 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:50:14.053082 3626 server_base.cc:1034] running on GCE node
I20260806 22:50:14.053289 3626 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:50:14.053512 3626 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:50:14.054648 3626 hybrid_clock.cc:648] HybridClock initialized: now 1786056614054633 us; error 38 us; skew 500 ppm
Aug 06 22:50:14 dist-test-slave-1xrh krb5kdc[3512](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056614, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.218.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:50:14.057184 3626 init.cc:377] Logged in from keytab as oryx/127.28.218.193@KRBTEST.COM (short username oryx)
I20260806 22:50:14.057610 3626 webserver.cc:460] Webserver started at http://127.28.218.193:42159/ using document root <none> and password file <none>
I20260806 22:50:14.057803 3626 fs_manager.cc:359] Metadata directory not provided
I20260806 22:50:14.057843 3626 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:50:14.057929 3626 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:50:14.058913 3626 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/ts-0/data/instance:
uuid: "7785c46d26114840970c8f163edf0196"
format_stamp: "Formatted at 2026-08-06 22:50:14 on dist-test-slave-1xrh"
server_key: "48d61b013e9f1bd16e158253001d95a7"
server_key_iv: "fcebbe89abf853e8145c5b7fd88a99c8"
server_key_version: "encryptionkey@0"
I20260806 22:50:14.059283 3626 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/ts-0/wal/instance:
uuid: "7785c46d26114840970c8f163edf0196"
format_stamp: "Formatted at 2026-08-06 22:50:14 on dist-test-slave-1xrh"
server_key: "48d61b013e9f1bd16e158253001d95a7"
server_key_iv: "fcebbe89abf853e8145c5b7fd88a99c8"
server_key_version: "encryptionkey@0"
I20260806 22:50:14.060580 3626 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:50:14.061300 3641 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:14.061492 3626 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:50:14.061571 3626 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/ts-0/wal
uuid: "7785c46d26114840970c8f163edf0196"
format_stamp: "Formatted at 2026-08-06 22:50:14 on dist-test-slave-1xrh"
server_key: "48d61b013e9f1bd16e158253001d95a7"
server_key_iv: "fcebbe89abf853e8145c5b7fd88a99c8"
server_key_version: "encryptionkey@0"
I20260806 22:50:14.061625 3626 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:50:14.074793 3626 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:50:14.076716 3626 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:50:14.076825 3626 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:50:14.076994 3626 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:50:14.077289 3626 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:50:14.077323 3626 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:14.077363 3626 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:50:14.077389 3626 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:14.083019 3626 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.193:35729
I20260806 22:50:14.083083 3759 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.193:35729 every 8 connection(s)
I20260806 22:50:14.083415 3626 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/ts-0/data/info.pb
I20260806 22:50:14.088511 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 3626
I20260806 22:50:14.088636 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/ts-0/wal/instance
I20260806 22:50:14.088848 29547 external_mini_cluster.cc:1428] Setting key 62fc312b14b531fb443fa8792a37bf8d
Aug 06 22:50:14 dist-test-slave-1xrh krb5kdc[3512](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056614, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.218.193@KRBTEST.COM for oryx/127.28.218.254@KRBTEST.COM
I20260806 22:50:14.092406 3620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:14.084144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.193:50441 (local address 127.28.218.254:44755)
0806 22:50:14.084217 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:14.084219 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:14.084355 (+ 136us) server_negotiation.cc:407] Connection header received
0806 22:50:14.084793 (+ 438us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:14.084794 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:14.084804 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:14.084851 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:14.085704 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:14.086211 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:14.087156 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:14.087358 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:14.087410 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:14.089769 (+ 2359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:50:14.089770 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:50:14.089770 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:50:14.089778 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:50:14.091067 (+ 1289us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:14.091481 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:14.091482 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:14.091482 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:14.091542 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:14.091781 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:14.091782 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:14.091783 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:14.091888 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:50:14.091972 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:14.092222 (+ 250us) server_negotiation.cc:299] Negotiation successful
0806 22:50:14.092318 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 22:50:14.092599 3764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:14.084098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:44755 (local address 127.28.218.193:50441)
0806 22:50:14.084247 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:14.084280 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:14.084716 (+ 436us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:14.084905 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:14.084908 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:14.085446 (+ 538us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:14.085628 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:14.085633 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:14.086285 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:14.086285 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:14.087053 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:14.087055 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:14.087425 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:14.087426 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:14.087519 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:14.087982 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:50:14.087989 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:14.089674 (+ 1685us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:50:14.091174 (+ 1500us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:14.091180 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:14.091182 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:14.091388 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:14.091610 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:14.091611 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:14.091611 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:14.091713 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:14.091965 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:50:14.091971 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:50:14.092097 (+ 126us) client_negotiation.cc:769] Sending connection context
0806 22:50:14.092199 (+ 102us) client_negotiation.cc:240] Negotiation successful
0806 22:50:14.092340 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":89,"threads_started":1}
I20260806 22:50:14.093147 3760 heartbeater.cc:347] Connected to a master server at 127.28.218.254:44755
I20260806 22:50:14.093255 3760 heartbeater.cc:464] Registering TS with master...
I20260806 22:50:14.093457 3760 heartbeater.cc:511] Master 127.28.218.254:44755 requested a full tablet report, sending...
I20260806 22:50:14.093950 3557 ts_manager.cc:194] Registered new tserver with Master: 7785c46d26114840970c8f163edf0196 (127.28.218.193:35729)
I20260806 22:50:14.094518 3557 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.28.218.193@KRBTEST.COM'} at 127.28.218.193:50441
WARNING: no policy specified for oryx/127.28.218.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.28.218.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.28.218.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/krb5kdc/oryx_127.28.218.194.keytab.
Entry for principal oryx/127.28.218.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/krb5kdc/oryx_127.28.218.194.keytab.
I20260806 22:50:14.116065 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.218.194:0
--local_ip_for_outbound_sockets=127.28.218.194
--webserver_interface=127.28.218.194
--webserver_port=0
--tserver_master_addrs=127.28.218.254:44755
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/krb5kdc/oryx_127.28.218.194.keytab
--principal=oryx/127.28.218.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.218.212:37527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:50:14.195845 3768 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:50:14.196038 3768 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:50:14.196059 3768 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:50:14.197118 3768 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:50:14.197151 3768 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:50:14.197176 3768 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.218.194
I20260806 22:50:14.198518 3768 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:37527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/krb5kdc/oryx_127.28.218.194.keytab
--principal=oryx/127.28.218.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.218.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.218.194
--webserver_port=0
--tserver_master_addrs=127.28.218.254:44755
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3768
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.218.194
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:50:14.198698 3768 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:50:14.198880 3768 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:50:14.203601 3776 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:14.203591 3774 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:14.203733 3773 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:50:14.203984 3768 server_base.cc:1034] running on GCE node
I20260806 22:50:14.204155 3768 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:50:14.204336 3768 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:50:14.205497 3768 hybrid_clock.cc:648] HybridClock initialized: now 1786056614205482 us; error 29 us; skew 500 ppm
Aug 06 22:50:14 dist-test-slave-1xrh krb5kdc[3512](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056614, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.218.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:50:14.208034 3768 init.cc:377] Logged in from keytab as oryx/127.28.218.194@KRBTEST.COM (short username oryx)
I20260806 22:50:14.208477 3768 webserver.cc:460] Webserver started at http://127.28.218.194:41751/ using document root <none> and password file <none>
I20260806 22:50:14.208701 3768 fs_manager.cc:359] Metadata directory not provided
I20260806 22:50:14.208792 3768 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:50:14.208945 3768 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:50:14.209939 3768 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/ts-1/data/instance:
uuid: "05b84cde108549378d6584ef921fed23"
format_stamp: "Formatted at 2026-08-06 22:50:14 on dist-test-slave-1xrh"
server_key: "717849e1ee3b8526f5950f6a20e59bbf"
server_key_iv: "041930f4fdf9054828ce2d5e4983d3cd"
server_key_version: "encryptionkey@0"
I20260806 22:50:14.210286 3768 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/ts-1/wal/instance:
uuid: "05b84cde108549378d6584ef921fed23"
format_stamp: "Formatted at 2026-08-06 22:50:14 on dist-test-slave-1xrh"
server_key: "717849e1ee3b8526f5950f6a20e59bbf"
server_key_iv: "041930f4fdf9054828ce2d5e4983d3cd"
server_key_version: "encryptionkey@0"
I20260806 22:50:14.211689 3768 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:50:14.212414 3783 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:14.212570 3768 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:50:14.212659 3768 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/ts-1/wal
uuid: "05b84cde108549378d6584ef921fed23"
format_stamp: "Formatted at 2026-08-06 22:50:14 on dist-test-slave-1xrh"
server_key: "717849e1ee3b8526f5950f6a20e59bbf"
server_key_iv: "041930f4fdf9054828ce2d5e4983d3cd"
server_key_version: "encryptionkey@0"
I20260806 22:50:14.212734 3768 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:50:14.224399 3768 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:50:14.226264 3768 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:50:14.226415 3768 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:50:14.226624 3768 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:50:14.226917 3768 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:50:14.226969 3768 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:14.227010 3768 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:50:14.227037 3768 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:14.232995 3768 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.194:45405
I20260806 22:50:14.233036 3901 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.194:45405 every 8 connection(s)
I20260806 22:50:14.233338 3768 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/ts-1/data/info.pb
Aug 06 22:50:14 dist-test-slave-1xrh krb5kdc[3512](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056614, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.218.194@KRBTEST.COM for oryx/127.28.218.254@KRBTEST.COM
I20260806 22:50:14.240978 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 3768
I20260806 22:50:14.241094 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/ts-1/wal/instance
I20260806 22:50:14.241338 29547 external_mini_cluster.cc:1428] Setting key 5b5263cbc411af0cdfbf25400acfb195
I20260806 22:50:14.242592 3620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:14.234019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.194:59967 (local address 127.28.218.254:44755)
0806 22:50:14.234062 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:14.234064 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:14.234269 (+ 205us) server_negotiation.cc:407] Connection header received
0806 22:50:14.234740 (+ 471us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:14.234741 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:14.234753 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:14.234853 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:14.235843 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:14.236362 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:14.237319 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:14.237538 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:14.237598 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:14.239674 (+ 2076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:50:14.239675 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:50:14.239675 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:50:14.239682 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:50:14.241049 (+ 1367us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:14.241579 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:14.241581 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:14.241581 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:14.241655 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:14.241930 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:14.241931 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:14.241932 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:14.242069 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:50:14.242141 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:14.242455 (+ 314us) server_negotiation.cc:299] Negotiation successful
0806 22:50:14.242528 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 22:50:14.242897 3904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:14.233969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:44755 (local address 127.28.218.194:59967)
0806 22:50:14.234121 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:14.234163 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:14.234636 (+ 473us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:14.234875 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:14.234878 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:14.235531 (+ 653us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:14.235722 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:14.235727 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:14.236453 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:14.236454 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:14.237233 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:14.237236 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:14.237615 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:14.237616 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:14.237715 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:14.238183 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:50:14.238190 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:14.239593 (+ 1403us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:50:14.241166 (+ 1573us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:14.241173 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:14.241175 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:14.241484 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:14.241720 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:14.241721 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:14.241721 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:14.241867 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:14.242143 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:50:14.242150 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:50:14.242313 (+ 163us) client_negotiation.cc:769] Sending connection context
0806 22:50:14.242435 (+ 122us) client_negotiation.cc:240] Negotiation successful
0806 22:50:14.242601 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":91,"threads_started":1}
I20260806 22:50:14.243427 3902 heartbeater.cc:347] Connected to a master server at 127.28.218.254:44755
I20260806 22:50:14.243520 3902 heartbeater.cc:464] Registering TS with master...
I20260806 22:50:14.243703 3902 heartbeater.cc:511] Master 127.28.218.254:44755 requested a full tablet report, sending...
I20260806 22:50:14.244089 3557 ts_manager.cc:194] Registered new tserver with Master: 05b84cde108549378d6584ef921fed23 (127.28.218.194:45405)
I20260806 22:50:14.244464 3557 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.28.218.194@KRBTEST.COM'} at 127.28.218.194:59967
WARNING: no policy specified for oryx/127.28.218.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.28.218.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.28.218.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/krb5kdc/oryx_127.28.218.195.keytab.
Entry for principal oryx/127.28.218.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/krb5kdc/oryx_127.28.218.195.keytab.
I20260806 22:50:14.270447 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.218.195:0
--local_ip_for_outbound_sockets=127.28.218.195
--webserver_interface=127.28.218.195
--webserver_port=0
--tserver_master_addrs=127.28.218.254:44755
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/krb5kdc/oryx_127.28.218.195.keytab
--principal=oryx/127.28.218.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.218.212:37527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:50:14.352761 3907 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:50:14.352979 3907 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:50:14.353004 3907 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:50:14.354069 3907 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:50:14.354103 3907 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:50:14.354130 3907 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.218.195
I20260806 22:50:14.355537 3907 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:37527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/krb5kdc/oryx_127.28.218.195.keytab
--principal=oryx/127.28.218.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.218.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.218.195
--webserver_port=0
--tserver_master_addrs=127.28.218.254:44755
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3907
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.218.195
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:50:14.355721 3907 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:50:14.355901 3907 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 22:50:14.360782 3907 server_base.cc:1034] running on GCE node
W20260806 22:50:14.360899 3916 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:14.360910 3913 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:14.360800 3914 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:50:14.361217 3907 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:50:14.361469 3907 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:50:14.362625 3907 hybrid_clock.cc:648] HybridClock initialized: now 1786056614362627 us; error 42 us; skew 500 ppm
Aug 06 22:50:14 dist-test-slave-1xrh krb5kdc[3512](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056614, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.218.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:50:14.365299 3907 init.cc:377] Logged in from keytab as oryx/127.28.218.195@KRBTEST.COM (short username oryx)
I20260806 22:50:14.365732 3907 webserver.cc:460] Webserver started at http://127.28.218.195:46163/ using document root <none> and password file <none>
I20260806 22:50:14.365952 3907 fs_manager.cc:359] Metadata directory not provided
I20260806 22:50:14.366017 3907 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:50:14.366168 3907 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:50:14.367296 3907 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/ts-2/data/instance:
uuid: "b11173b0207c46549ca94b7a987905cf"
format_stamp: "Formatted at 2026-08-06 22:50:14 on dist-test-slave-1xrh"
server_key: "793cf81257c3498a570ee9636982688f"
server_key_iv: "bd7cd6aaef60bce3ab4194010109b3d9"
server_key_version: "encryptionkey@0"
I20260806 22:50:14.367658 3907 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/ts-2/wal/instance:
uuid: "b11173b0207c46549ca94b7a987905cf"
format_stamp: "Formatted at 2026-08-06 22:50:14 on dist-test-slave-1xrh"
server_key: "793cf81257c3498a570ee9636982688f"
server_key_iv: "bd7cd6aaef60bce3ab4194010109b3d9"
server_key_version: "encryptionkey@0"
I20260806 22:50:14.368997 3907 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 22:50:14.369720 3923 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:14.369894 3907 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:50:14.369995 3907 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/ts-2/wal
uuid: "b11173b0207c46549ca94b7a987905cf"
format_stamp: "Formatted at 2026-08-06 22:50:14 on dist-test-slave-1xrh"
server_key: "793cf81257c3498a570ee9636982688f"
server_key_iv: "bd7cd6aaef60bce3ab4194010109b3d9"
server_key_version: "encryptionkey@0"
I20260806 22:50:14.370064 3907 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:50:14.430478 3907 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:50:14.432626 3907 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:50:14.432801 3907 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:50:14.433071 3907 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:50:14.433413 3907 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:50:14.433467 3907 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:14.433514 3907 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:50:14.433549 3907 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:14.439563 3907 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.195:42091
I20260806 22:50:14.439607 4043 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.195:42091 every 8 connection(s)
I20260806 22:50:14.439949 3907 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/ts-2/data/info.pb
Aug 06 22:50:14 dist-test-slave-1xrh krb5kdc[3512](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056614, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.218.195@KRBTEST.COM for oryx/127.28.218.254@KRBTEST.COM
I20260806 22:50:14.446570 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 3907
I20260806 22:50:14.446683 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/ts-2/wal/instance
I20260806 22:50:14.446873 29547 external_mini_cluster.cc:1428] Setting key 5316d2387de963a07d24c34943a842a5
I20260806 22:50:14.449437 3620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:14.440650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.195:36245 (local address 127.28.218.254:44755)
0806 22:50:14.440715 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:14.440717 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:14.440922 (+ 205us) server_negotiation.cc:407] Connection header received
0806 22:50:14.441488 (+ 566us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:14.441489 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:14.441499 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:14.441575 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:14.442556 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:14.443048 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:14.444137 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:14.444315 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:14.444367 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:14.446496 (+ 2129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:50:14.446498 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:50:14.446498 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:50:14.446507 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:50:14.448225 (+ 1718us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:14.448605 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:14.448605 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:14.448606 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:14.448661 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:14.448909 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:14.448910 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:14.448910 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:14.449028 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:50:14.449079 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:14.449310 (+ 231us) server_negotiation.cc:299] Negotiation successful
0806 22:50:14.449374 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 22:50:14.449625 4047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:14.440617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:44755 (local address 127.28.218.195:36245)
0806 22:50:14.440789 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:14.440832 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:14.441396 (+ 564us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:14.441667 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:14.441671 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:14.442274 (+ 603us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:14.442475 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:14.442480 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:14.443204 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:14.443205 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:14.444058 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:14.444061 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:14.444386 (+ 325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:14.444386 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:14.444491 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:14.445050 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:50:14.445064 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:14.446405 (+ 1341us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:50:14.448282 (+ 1877us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:14.448287 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:14.448289 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:14.448517 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:14.448717 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:14.448718 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:14.448718 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:14.448821 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:14.449081 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:50:14.449085 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:50:14.449200 (+ 115us) client_negotiation.cc:769] Sending connection context
0806 22:50:14.449282 (+ 82us) client_negotiation.cc:240] Negotiation successful
0806 22:50:14.449402 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":102,"threads_started":1}
I20260806 22:50:14.450067 4044 heartbeater.cc:347] Connected to a master server at 127.28.218.254:44755
I20260806 22:50:14.450162 4044 heartbeater.cc:464] Registering TS with master...
I20260806 22:50:14.450342 4044 heartbeater.cc:511] Master 127.28.218.254:44755 requested a full tablet report, sending...
I20260806 22:50:14.450701 3557 ts_manager.cc:194] Registered new tserver with Master: b11173b0207c46549ca94b7a987905cf (127.28.218.195:42091)
I20260806 22:50:14.451057 3557 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.28.218.195@KRBTEST.COM'} at 127.28.218.195:36245
I20260806 22:50:14.458653 29547 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.28.218.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.28.218.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.28.218.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/krb5kdc/oryx_127.28.218.253.keytab.
Entry for principal oryx/127.28.218.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/krb5kdc/oryx_127.28.218.253.keytab.
I20260806 22:50:14.486785 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.218.253:39243
--webserver_interface=127.28.218.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/krb5kdc/oryx_127.28.218.253.keytab
--principal=oryx/127.28.218.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.218.212:37527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.218.254:44755,127.28.218.253:39243
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:50:14.569873 4050 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:50:14.570087 4050 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:50:14.570109 4050 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:50:14.571303 4050 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:50:14.571353 4050 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:50:14.571368 4050 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:50:14.571412 4050 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:50:14.571435 4050 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:50:14.573014 4050 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:37527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.28.218.254:44755,127.28.218.253:39243
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/krb5kdc/oryx_127.28.218.253.keytab
--principal=oryx/127.28.218.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.218.253:39243
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.28.218.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4050
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
W20260806 22:50:14.573143 4050 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.28.218.254:44755,127.28.218.253:39243'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260806 22:50:14.573292 4050 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:50:14.573522 4050 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:50:14.578536 4058 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:14.578527 4056 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:14.578502 4055 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:50:14.578599 4050 server_base.cc:1034] running on GCE node
I20260806 22:50:14.578974 4050 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:50:14.579332 4050 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:50:14.580492 4050 hybrid_clock.cc:648] HybridClock initialized: now 1786056614580478 us; error 32 us; skew 500 ppm
Aug 06 22:50:14 dist-test-slave-1xrh krb5kdc[3512](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056614, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.218.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:50:14.586175 4050 init.cc:377] Logged in from keytab as oryx/127.28.218.253@KRBTEST.COM (short username oryx)
I20260806 22:50:14.586681 4050 webserver.cc:460] Webserver started at http://127.28.218.253:41987/ using document root <none> and password file <none>
I20260806 22:50:14.586910 4050 fs_manager.cc:359] Metadata directory not provided
I20260806 22:50:14.586975 4050 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:50:14.587167 4050 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:50:14.588240 4050 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/master-1/data/instance:
uuid: "eb135056735a446dbff018e4072cab0c"
format_stamp: "Formatted at 2026-08-06 22:50:14 on dist-test-slave-1xrh"
server_key: "20b7fb5970980ec2b957ef3d23350dc1"
server_key_iv: "843e66cc9f035db3cc9b60f8014c8609"
server_key_version: "encryptionkey@0"
I20260806 22:50:14.588608 4050 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/master-1/wal/instance:
uuid: "eb135056735a446dbff018e4072cab0c"
format_stamp: "Formatted at 2026-08-06 22:50:14 on dist-test-slave-1xrh"
server_key: "20b7fb5970980ec2b957ef3d23350dc1"
server_key_iv: "843e66cc9f035db3cc9b60f8014c8609"
server_key_version: "encryptionkey@0"
I20260806 22:50:14.589932 4050 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:50:14.590668 4065 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:14.590855 4050 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:14.590950 4050 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/master-1/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/master-1/wal
uuid: "eb135056735a446dbff018e4072cab0c"
format_stamp: "Formatted at 2026-08-06 22:50:14 on dist-test-slave-1xrh"
server_key: "20b7fb5970980ec2b957ef3d23350dc1"
server_key_iv: "843e66cc9f035db3cc9b60f8014c8609"
server_key_version: "encryptionkey@0"
I20260806 22:50:14.591025 4050 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:50:14.606277 4050 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:50:14.608255 4050 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:50:14.608418 4050 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:50:14.612550 4050 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.253:39243
I20260806 22:50:14.612584 4118 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.253:39243 every 8 connection(s)
I20260806 22:50:14.612960 4050 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/master-1/data/info.pb
I20260806 22:50:14.613925 4119 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:50:14.615613 4119 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.218.254" port: 44755 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 22:50:14 dist-test-slave-1xrh krb5kdc[3512](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056614, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.218.253@KRBTEST.COM for oryx/127.28.218.254@KRBTEST.COM
I20260806 22:50:14.622645 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 4050
I20260806 22:50:14.624548 3620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:14.616320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51792 (local address 127.28.218.254:44755)
0806 22:50:14.616410 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:14.616412 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:14.616584 (+ 172us) server_negotiation.cc:407] Connection header received
0806 22:50:14.617027 (+ 443us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:14.617028 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:14.617038 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:14.617092 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:14.617666 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:14.618169 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:14.619107 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:14.619329 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:14.619392 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:14.621438 (+ 2046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:50:14.621439 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:50:14.621439 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:50:14.621446 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:50:14.622799 (+ 1353us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:14.623328 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:14.623329 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:14.623329 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:14.623415 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:14.623701 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:14.623702 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:14.623702 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:14.623869 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:50:14.623927 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:14.624363 (+ 436us) server_negotiation.cc:299] Negotiation successful
0806 22:50:14.624453 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 22:50:14.624617 4121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:14.616276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:44755 (local address 127.0.0.1:51792)
0806 22:50:14.616466 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:14.616504 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:14.616943 (+ 439us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:14.617161 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:14.617164 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:14.617410 (+ 246us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:14.617596 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:14.617599 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:14.618239 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:14.618240 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:14.619028 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:14.619030 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:14.619417 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:14.619418 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:14.619531 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:14.619974 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:50:14.619982 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:14.621337 (+ 1355us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:50:14.622874 (+ 1537us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:14.622880 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:14.622881 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:14.623235 (+ 354us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:14.623490 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:14.623490 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:14.623491 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:14.623632 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:14.623987 (+ 355us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:50:14.623995 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:50:14.624163 (+ 168us) client_negotiation.cc:769] Sending connection context
0806 22:50:14.624291 (+ 128us) client_negotiation.cc:240] Negotiation successful
0806 22:50:14.624431 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":110,"threads_started":1}
I20260806 22:50:14.625378 4119 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.218.253" port: 39243 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 22:50:14 dist-test-slave-1xrh krb5kdc[3512](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056613, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.28.218.253@KRBTEST.COM
Aug 06 22:50:14 dist-test-slave-1xrh krb5kdc[3512](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056614, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.218.253@KRBTEST.COM for oryx/127.28.218.253@KRBTEST.COM
I20260806 22:50:14.633127 4123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:14.623224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46356 (local address 127.28.218.253:39243)
0806 22:50:14.623393 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:14.623400 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:14.623424 (+ 24us) server_negotiation.cc:407] Connection header received
0806 22:50:14.623504 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:14.623506 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:14.623519 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:14.623583 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:14.624315 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:14.625290 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:14.626169 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:14.626485 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:14.626573 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:14.629124 (+ 2551us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:50:14.629132 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:50:14.629132 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:50:14.629139 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:50:14.631337 (+ 2198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:14.631801 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:14.631804 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:14.631804 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:14.631869 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:14.632085 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:14.632086 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:14.632086 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:14.632254 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:50:14.632383 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:14.632873 (+ 490us) server_negotiation.cc:299] Negotiation successful
0806 22:50:14.632973 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":82,"threads_started":1}
I20260806 22:50:14.633261 4124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:14.625705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46368 (local address 127.28.218.253:39243)
0806 22:50:14.625859 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:14.625860 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:14.625866 (+ 6us) server_negotiation.cc:407] Connection header received
0806 22:50:14.625879 (+ 13us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:14.625879 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:14.625886 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:14.625983 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:14.626242 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:14.626702 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:14.627403 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:14.627606 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:14.627682 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:14.629473 (+ 1791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:50:14.629474 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:50:14.629474 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:50:14.629480 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:50:14.631275 (+ 1795us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:14.632314 (+ 1039us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:14.632315 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:14.632316 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:14.632390 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:14.632592 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:14.632593 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:14.632593 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:14.632726 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:50:14.632786 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:14.632973 (+ 187us) server_negotiation.cc:299] Negotiation successful
0806 22:50:14.633041 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":94,"threads_started":1}
I20260806 22:50:14.633476 4121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:14.625643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.253:39243 (local address 127.0.0.1:46368)
0806 22:50:14.625687 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:14.625697 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:14.625733 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:14.625988 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:14.625989 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:14.626126 (+ 137us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:14.626166 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:14.626167 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:14.626773 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:14.626774 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:14.627319 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:14.627320 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:14.627682 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:14.627683 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:14.627743 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:14.628119 (+ 376us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:50:14.628124 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:14.629398 (+ 1274us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:50:14.632004 (+ 2606us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:14.632005 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:14.632005 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:14.632219 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:14.632453 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:14.632454 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:14.632454 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:14.632535 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:14.632808 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:50:14.632808 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:50:14.632849 (+ 41us) client_negotiation.cc:769] Sending connection context
0806 22:50:14.632907 (+ 58us) client_negotiation.cc:240] Negotiation successful
0806 22:50:14.633362 (+ 455us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29,"mutex_wait_us":365}
I20260806 22:50:14.635807 4119 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb135056735a446dbff018e4072cab0c: Bootstrap starting.
I20260806 22:50:14.636631 4119 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eb135056735a446dbff018e4072cab0c: Neither blocks nor log segments found. Creating new log.
I20260806 22:50:14.636958 4119 log.cc:824] T 00000000000000000000000000000000 P eb135056735a446dbff018e4072cab0c: Log is configured to *not* fsync() on all Append() calls
I20260806 22:50:14.637916 4119 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb135056735a446dbff018e4072cab0c: No bootstrap required, opened a new log
I20260806 22:50:14.640266 4119 raft_consensus.cc:348] T 00000000000000000000000000000000 P eb135056735a446dbff018e4072cab0c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20186b499bd34a7d9d497ba785db0fc9" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 44755 } } peers { permanent_uuid: "eb135056735a446dbff018e4072cab0c" member_type: VOTER last_known_addr { host: "127.28.218.253" port: 39243 } }
I20260806 22:50:14.640443 4119 raft_consensus.cc:374] T 00000000000000000000000000000000 P eb135056735a446dbff018e4072cab0c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:50:14.640493 4119 raft_consensus.cc:729] T 00000000000000000000000000000000 P eb135056735a446dbff018e4072cab0c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb135056735a446dbff018e4072cab0c, State: Initialized, Role: FOLLOWER
I20260806 22:50:14.640617 4119 consensus_queue.cc:260] T 00000000000000000000000000000000 P eb135056735a446dbff018e4072cab0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20186b499bd34a7d9d497ba785db0fc9" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 44755 } } peers { permanent_uuid: "eb135056735a446dbff018e4072cab0c" member_type: VOTER last_known_addr { host: "127.28.218.253" port: 39243 } }
I20260806 22:50:14.640882 4126 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb135056735a446dbff018e4072cab0c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20186b499bd34a7d9d497ba785db0fc9" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 44755 } } peers { permanent_uuid: "eb135056735a446dbff018e4072cab0c" member_type: VOTER last_known_addr { host: "127.28.218.253" port: 39243 } } }
I20260806 22:50:14.640944 4119 sys_catalog.cc:564] T 00000000000000000000000000000000 P eb135056735a446dbff018e4072cab0c [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:50:14.640970 4126 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb135056735a446dbff018e4072cab0c [sys.catalog]: This master's current role is: FOLLOWER
W20260806 22:50:14.642838 4139 catalog_manager.cc:1558] T 00000000000000000000000000000000 P eb135056735a446dbff018e4072cab0c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 22:50:14.642953 4139 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 22:50:14.645160 4050 master_runner.cc:428] Detected that this master eb135056735a446dbff018e4072cab0c is joining an existing cluster
I20260806 22:50:14.645246 4050 master_runner.cc:432] Initiating AddMaster RPC to add 127.28.218.253:39243
I20260806 22:50:14.654551 3620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:14.646385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51806 (local address 127.28.218.254:44755)
0806 22:50:14.646457 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:14.646459 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:14.647390 (+ 931us) server_negotiation.cc:407] Connection header received
0806 22:50:14.647460 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:14.647461 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:14.647476 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:14.647544 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:14.648107 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:14.648941 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:14.649676 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:14.649930 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:14.650003 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:14.651247 (+ 1244us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:50:14.651249 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:50:14.651249 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:50:14.651258 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:50:14.653038 (+ 1780us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:14.653604 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:14.653605 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:14.653606 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:14.653669 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:14.653899 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:14.653900 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:14.653901 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:14.654039 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:50:14.654102 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:14.654329 (+ 227us) server_negotiation.cc:299] Negotiation successful
0806 22:50:14.654426 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260806 22:50:14.654685 4124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:14.646436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46380 (local address 127.28.218.253:39243)
0806 22:50:14.646475 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:14.646477 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:14.647532 (+ 1055us) server_negotiation.cc:407] Connection header received
0806 22:50:14.647596 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:14.647596 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:14.647606 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:14.647664 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:14.648320 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:14.649052 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:14.650075 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:14.650375 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:14.650427 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:14.651488 (+ 1061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:50:14.651489 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:50:14.651490 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:50:14.651495 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:50:14.653540 (+ 2045us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:14.653921 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:14.653921 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:14.653922 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:14.653972 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:14.654164 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:14.654165 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:14.654165 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:14.654277 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:50:14.654315 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:14.654503 (+ 188us) server_negotiation.cc:299] Negotiation successful
0806 22:50:14.654592 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 22:50:14.654898 4147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:14.646338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:44755 (local address 127.0.0.1:51806)
0806 22:50:14.647275 (+ 937us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:14.647287 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:14.647396 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:14.647612 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:14.647613 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:14.647880 (+ 267us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:14.647962 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:14.647965 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:14.649043 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:14.649044 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:14.649592 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:14.649593 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:14.650062 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:14.650063 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:14.650112 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:14.650578 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:50:14.650583 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:14.651093 (+ 510us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:50:14.653139 (+ 2046us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:14.653140 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:14.653140 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:14.653512 (+ 372us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:14.653743 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:14.653743 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:14.653743 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:14.653823 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:14.654155 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:50:14.654156 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:50:14.654204 (+ 48us) client_negotiation.cc:769] Sending connection context
0806 22:50:14.654646 (+ 442us) client_negotiation.cc:240] Negotiation successful
0806 22:50:14.654773 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":913,"thread_start_us":113,"threads_started":1}
I20260806 22:50:14.655539 4146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:14.646341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.253:39243 (local address 127.0.0.1:46380)
0806 22:50:14.647485 (+ 1144us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:14.647494 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:14.647541 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:14.647683 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:14.647684 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:14.648155 (+ 471us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:14.648217 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:14.648235 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:14.649161 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:14.649161 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:14.649983 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:14.649985 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:14.650493 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:14.650494 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:14.650553 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:14.650951 (+ 398us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:50:14.650956 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:14.651364 (+ 408us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:50:14.653624 (+ 2260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:14.653624 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:14.653625 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:14.653836 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:14.654052 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:14.654053 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:14.654053 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:14.654103 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:14.654353 (+ 250us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:50:14.654353 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:50:14.654396 (+ 43us) client_negotiation.cc:769] Sending connection context
0806 22:50:14.655330 (+ 934us) client_negotiation.cc:240] Negotiation successful
0806 22:50:14.655441 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1129,"thread_start_us":99,"threads_started":1}
Aug 06 22:50:14 dist-test-slave-1xrh krb5kdc[3512](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056613, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.218.254@KRBTEST.COM for oryx/127.28.218.253@KRBTEST.COM
I20260806 22:50:14.666895 4124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:14.657768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46382 (local address 127.28.218.253:39243)
0806 22:50:14.657804 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:14.657806 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:14.657905 (+ 99us) server_negotiation.cc:407] Connection header received
0806 22:50:14.658037 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:14.658038 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:14.658048 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:14.658076 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:14.658662 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:14.659411 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:14.660535 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:14.660733 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:14.660834 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:14.663416 (+ 2582us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:50:14.663418 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:50:14.663418 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:50:14.663425 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:50:14.665564 (+ 2139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:14.666081 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:14.666082 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:14.666083 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:14.666139 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:14.666352 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:14.666353 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:14.666354 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:14.666467 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:50:14.666540 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:14.666717 (+ 177us) server_negotiation.cc:299] Negotiation successful
0806 22:50:14.666797 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 22:50:14.666918 4148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:14.657582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.253:39243 (local address 127.0.0.1:46382)
0806 22:50:14.657717 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:14.657851 (+ 134us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:14.657941 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:14.658150 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:14.658152 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:14.658506 (+ 354us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:14.658567 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:14.658570 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:14.659507 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:14.659507 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:14.660449 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:14.660451 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:14.660837 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:14.660838 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:14.660941 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:14.661492 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:50:14.661497 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:14.663284 (+ 1787us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:50:14.665690 (+ 2406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:14.665697 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:14.665700 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:14.665973 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:14.666220 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:14.666221 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:14.666221 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:14.666286 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:14.666532 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:50:14.666539 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:50:14.666593 (+ 54us) client_negotiation.cc:769] Sending connection context
0806 22:50:14.666688 (+ 95us) client_negotiation.cc:240] Negotiation successful
0806 22:50:14.666809 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":90,"thread_start_us":53,"threads_started":1}
I20260806 22:50:14.675643 3557 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.28.218.253:39243: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "eb135056735a446dbff018e4072cab0c"
member_type: NON_VOTER
last_known_addr {
host: "127.28.218.253"
port: 39243
}
attrs {
promote: true
}
}
dest_uuid: "20186b499bd34a7d9d497ba785db0fc9"
cas_config_opid_index: -1
I20260806 22:50:14.676079 3557 consensus_queue.cc:237] T 00000000000000000000000000000000 P 20186b499bd34a7d9d497ba785db0fc9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "20186b499bd34a7d9d497ba785db0fc9" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 44755 } } peers { permanent_uuid: "eb135056735a446dbff018e4072cab0c" member_type: NON_VOTER last_known_addr { host: "127.28.218.253" port: 39243 } attrs { promote: true } }
I20260806 22:50:14.676604 4150 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 20186b499bd34a7d9d497ba785db0fc9 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER eb135056735a446dbff018e4072cab0c (127.28.218.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "20186b499bd34a7d9d497ba785db0fc9" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 44755 } } peers { permanent_uuid: "eb135056735a446dbff018e4072cab0c" member_type: NON_VOTER last_known_addr { host: "127.28.218.253" port: 39243 } attrs { promote: true } } }
I20260806 22:50:14.676879 4092 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eb135056735a446dbff018e4072cab0c [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:50:14.677846 4150 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.28.218.253:39243
I20260806 22:50:14.678052 4151 sys_catalog.cc:455] T 00000000000000000000000000000000 P 20186b499bd34a7d9d497ba785db0fc9 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "20186b499bd34a7d9d497ba785db0fc9" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "20186b499bd34a7d9d497ba785db0fc9" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 44755 } } peers { permanent_uuid: "eb135056735a446dbff018e4072cab0c" member_type: NON_VOTER last_known_addr { host: "127.28.218.253" port: 39243 } attrs { promote: true } } }
I20260806 22:50:14.678126 4151 sys_catalog.cc:458] T 00000000000000000000000000000000 P 20186b499bd34a7d9d497ba785db0fc9 [sys.catalog]: This master's current role is: LEADER
I20260806 22:50:14.678166 4050 master.cc:537] Master@127.28.218.253:39243 shutting down...
I20260806 22:50:14.678151 4092 raft_consensus.cc:1261] T 00000000000000000000000000000000 P eb135056735a446dbff018e4072cab0c [term 1 FOLLOWER]: Refusing update from remote peer 20186b499bd34a7d9d497ba785db0fc9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260806 22:50:14.678655 4149 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 20186b499bd34a7d9d497ba785db0fc9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb135056735a446dbff018e4072cab0c" member_type: NON_VOTER last_known_addr { host: "127.28.218.253" port: 39243 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 22:50:14.679458 3543 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260806 22:50:14.679970 3543 consensus_peers.cc:487] T 00000000000000000000000000000000 P 20186b499bd34a7d9d497ba785db0fc9 -> Peer eb135056735a446dbff018e4072cab0c (127.28.218.253:39243): Couldn't send request to peer eb135056735a446dbff018e4072cab0c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260806 22:50:14.680995 4050 raft_consensus.cc:2229] T 00000000000000000000000000000000 P eb135056735a446dbff018e4072cab0c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:50:14.681088 4050 raft_consensus.cc:2258] T 00000000000000000000000000000000 P eb135056735a446dbff018e4072cab0c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:50:14.681143 4050 tablet_replica.cc:330] T 00000000000000000000000000000000 P eb135056735a446dbff018e4072cab0c: stopping tablet replica
I20260806 22:50:14.693818 4050 master.cc:552] Master@127.28.218.253:39243 shutdown complete.
I20260806 22:50:14.693876 4050 master_runner.cc:305] Clearing existing system tablet
I20260806 22:50:14.694013 4050 fs_manager.cc:359] Metadata directory not provided
I20260806 22:50:14.694062 4050 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:50:14.694788 4050 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.001s
I20260806 22:50:14.695315 4155 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:14.695459 4050 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:50:14.695521 4050 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:14.695545 4050 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/master-1/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/master-1/wal
uuid: "eb135056735a446dbff018e4072cab0c"
format_stamp: "Formatted at 2026-08-06 22:50:14 on dist-test-slave-1xrh"
server_key: "20b7fb5970980ec2b957ef3d23350dc1"
server_key_iv: "843e66cc9f035db3cc9b60f8014c8609"
server_key_version: "encryptionkey@0"
I20260806 22:50:14.696400 4050 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P eb135056735a446dbff018e4072cab0c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:50:14.698287 4050 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P eb135056735a446dbff018e4072cab0c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260806 22:50:14.698374 4050 log.cc:1196] T 00000000000000000000000000000000 P eb135056735a446dbff018e4072cab0c: Deleting WAL directory at /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260806 22:50:14.698596 4050 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P eb135056735a446dbff018e4072cab0c: Deleting consensus metadata
I20260806 22:50:14.698719 4050 master_runner.cc:315] Copying system tablet from 127.28.218.254:44755
I20260806 22:50:14.699493 4050 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P eb135056735a446dbff018e4072cab0c: tablet copy: Beginning tablet copy session from remote peer at address 127.28.218.254:44755
I20260806 22:50:14.706137 4161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:14.699867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:44755 (local address 127.0.0.1:51820)
0806 22:50:14.700050 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:14.700072 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:14.700128 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:14.700271 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:14.700271 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:14.700447 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:14.700722 (+ 275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:14.700724 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:14.701685 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:14.701685 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:14.702235 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:14.702236 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:14.702705 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:14.702706 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:14.702745 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:14.703243 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:50:14.703248 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:14.703568 (+ 320us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:50:14.705247 (+ 1679us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:14.705247 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:14.705248 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:14.705408 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:14.705606 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:14.705606 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:14.705606 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:14.705655 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:14.705901 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:50:14.705902 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:50:14.705940 (+ 38us) client_negotiation.cc:769] Sending connection context
0806 22:50:14.705987 (+ 47us) client_negotiation.cc:240] Negotiation successful
0806 22:50:14.706063 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":99,"threads_started":1}
I20260806 22:50:14.706204 3620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:14.699934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51820 (local address 127.28.218.254:44755)
0806 22:50:14.700000 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:14.700003 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:14.700133 (+ 130us) server_negotiation.cc:407] Connection header received
0806 22:50:14.700193 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:14.700194 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:14.700206 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:14.700267 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:14.700800 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:14.701608 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:14.702311 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:14.702634 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:14.702706 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:14.703645 (+ 939us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:50:14.703647 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:50:14.703647 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:50:14.703654 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:50:14.705159 (+ 1505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:14.705479 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:14.705480 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:14.705480 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:14.705540 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:14.705709 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:14.705710 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:14.705710 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:14.705839 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:50:14.705900 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:14.706027 (+ 127us) server_negotiation.cc:299] Negotiation successful
0806 22:50:14.706108 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 22:50:14.706547 3582 tablet_copy_service.cc:140] P 20186b499bd34a7d9d497ba785db0fc9: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer eb135056735a446dbff018e4072cab0c ({username='oryx', principal='oryx/127.28.218.253@KRBTEST.COM'} at 127.0.0.1:51820)
I20260806 22:50:14.706696 3582 tablet_copy_service.cc:161] P 20186b499bd34a7d9d497ba785db0fc9: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer eb135056735a446dbff018e4072cab0c at {username='oryx', principal='oryx/127.28.218.253@KRBTEST.COM'} at 127.0.0.1:51820: session id = eb135056735a446dbff018e4072cab0c-00000000000000000000000000000000
I20260806 22:50:14.707523 3582 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 20186b499bd34a7d9d497ba785db0fc9: Tablet Copy: opened 0 blocks and 1 log segments
I20260806 22:50:14.708104 4050 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260806 22:50:14.709406 4050 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P eb135056735a446dbff018e4072cab0c: tablet copy: Starting download of 0 data blocks...
I20260806 22:50:14.709559 4050 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P eb135056735a446dbff018e4072cab0c: tablet copy: Starting download of 1 WAL segments...
I20260806 22:50:14.710491 4050 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P eb135056735a446dbff018e4072cab0c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260806 22:50:14.711396 3582 tablet_copy_service.cc:342] P 20186b499bd34a7d9d497ba785db0fc9: Request end of tablet copy session eb135056735a446dbff018e4072cab0c-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.28.218.253@KRBTEST.COM'} at 127.0.0.1:51820
I20260806 22:50:14.711455 3582 tablet_copy_service.cc:433] P 20186b499bd34a7d9d497ba785db0fc9: ending tablet copy session eb135056735a446dbff018e4072cab0c-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer eb135056735a446dbff018e4072cab0c
I20260806 22:50:14.714037 4050 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:50:14.715240 4165 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:14.715269 4163 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:14.715251 4162 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:50:14.715572 4050 server_base.cc:1034] running on GCE node
I20260806 22:50:14.715659 4050 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:50:14.715790 4050 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:50:14.716915 4050 hybrid_clock.cc:648] HybridClock initialized: now 1786056614716903 us; error 32 us; skew 500 ppm
Aug 06 22:50:14 dist-test-slave-1xrh krb5kdc[3512](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056614, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.218.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:50:14.718251 4050 init.cc:377] Logged in from keytab as oryx/127.28.218.253@KRBTEST.COM (short username oryx)
I20260806 22:50:14.718557 4050 webserver.cc:460] Webserver started at http://127.28.218.253:36249/ using document root <none> and password file <none>
I20260806 22:50:14.718686 4050 fs_manager.cc:359] Metadata directory not provided
I20260806 22:50:14.718730 4050 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:50:14.719429 4050 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:50:14.719974 4172 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:14.720112 4050 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:50:14.720175 4050 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/master-1/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/master-1/wal
uuid: "eb135056735a446dbff018e4072cab0c"
format_stamp: "Formatted at 2026-08-06 22:50:14 on dist-test-slave-1xrh"
server_key: "20b7fb5970980ec2b957ef3d23350dc1"
server_key_iv: "843e66cc9f035db3cc9b60f8014c8609"
server_key_version: "encryptionkey@0"
I20260806 22:50:14.720256 4050 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:50:14.728612 4050 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:50:14.730268 4050 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:50:14.733074 4050 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.253:39243
I20260806 22:50:14.733484 4227 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.253:39243 every 8 connection(s)
I20260806 22:50:14.733526 4050 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/master-1/data/info.pb
I20260806 22:50:14.734486 4228 sys_catalog.cc:263] Verifying existing consensus state
I20260806 22:50:14.735016 4228 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb135056735a446dbff018e4072cab0c: Bootstrap starting.
I20260806 22:50:14.738210 4228 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb135056735a446dbff018e4072cab0c: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 22:50:14.738533 4228 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb135056735a446dbff018e4072cab0c: Bootstrap complete.
I20260806 22:50:14.738762 4228 raft_consensus.cc:348] T 00000000000000000000000000000000 P eb135056735a446dbff018e4072cab0c [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "20186b499bd34a7d9d497ba785db0fc9" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 44755 } } peers { permanent_uuid: "eb135056735a446dbff018e4072cab0c" member_type: NON_VOTER last_known_addr { host: "127.28.218.253" port: 39243 } attrs { promote: true } }
I20260806 22:50:14.738909 4228 raft_consensus.cc:729] T 00000000000000000000000000000000 P eb135056735a446dbff018e4072cab0c [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: eb135056735a446dbff018e4072cab0c, State: Initialized, Role: LEARNER
I20260806 22:50:14.738957 4228 consensus_queue.cc:260] T 00000000000000000000000000000000 P eb135056735a446dbff018e4072cab0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "20186b499bd34a7d9d497ba785db0fc9" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 44755 } } peers { permanent_uuid: "eb135056735a446dbff018e4072cab0c" member_type: NON_VOTER last_known_addr { host: "127.28.218.253" port: 39243 } attrs { promote: true } }
I20260806 22:50:14.739192 4228 sys_catalog.cc:564] T 00000000000000000000000000000000 P eb135056735a446dbff018e4072cab0c [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:50:14.739228 4232 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb135056735a446dbff018e4072cab0c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "20186b499bd34a7d9d497ba785db0fc9" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 44755 } } peers { permanent_uuid: "eb135056735a446dbff018e4072cab0c" member_type: NON_VOTER last_known_addr { host: "127.28.218.253" port: 39243 } attrs { promote: true } } }
I20260806 22:50:14.739459 4232 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb135056735a446dbff018e4072cab0c [sys.catalog]: This master's current role is: LEARNER
I20260806 22:50:14.741005 4244 catalog_manager.cc:1259] Loaded cluster ID: 49a945fc1aed48f391333d0beb7d3f21
I20260806 22:50:14.741071 4244 catalog_manager.cc:1552] T 00000000000000000000000000000000 P eb135056735a446dbff018e4072cab0c: loading cluster ID for follower catalog manager: success
I20260806 22:50:14.742177 4244 catalog_manager.cc:1574] T 00000000000000000000000000000000 P eb135056735a446dbff018e4072cab0c: acquiring CA information for follower catalog manager: success
I20260806 22:50:14.742477 4244 catalog_manager.cc:1602] T 00000000000000000000000000000000 P eb135056735a446dbff018e4072cab0c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 22:50:14.742977 4230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:14.735402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46402 (local address 127.28.218.253:39243)
0806 22:50:14.735534 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:14.735536 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:14.735544 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:50:14.735571 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:14.735572 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:14.735580 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:14.735657 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:14.736205 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:14.736989 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:14.737708 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:14.737916 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:14.737990 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:14.739624 (+ 1634us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:50:14.739625 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:50:14.739626 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:50:14.739631 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:50:14.741470 (+ 1839us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:14.742036 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:14.742037 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:14.742037 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:14.742118 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:14.742336 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:14.742336 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:14.742337 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:14.742442 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:50:14.742556 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:14.742653 (+ 97us) server_negotiation.cc:299] Negotiation successful
0806 22:50:14.742741 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":91,"threads_started":1}
WARNING: no policy specified for oryx/127.28.218.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.28.218.252@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.28.218.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/krb5kdc/oryx_127.28.218.252.keytab.
Entry for principal oryx/127.28.218.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/krb5kdc/oryx_127.28.218.252.keytab.
I20260806 22:50:14.774802 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.218.252:39503
--webserver_interface=127.28.218.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/krb5kdc/oryx_127.28.218.252.keytab
--principal=oryx/127.28.218.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.218.212:37527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.218.254:44755,127.28.218.253:39243,127.28.218.252:39503
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:50:14.862920 4247 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:50:14.863211 4247 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:50:14.863260 4247 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:50:14.864638 4247 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:50:14.864714 4247 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:50:14.864735 4247 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:50:14.864774 4247 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:50:14.864794 4247 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:50:14.866551 4247 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:37527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.28.218.254:44755,127.28.218.253:39243,127.28.218.252:39503
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/krb5kdc/oryx_127.28.218.252.keytab
--principal=oryx/127.28.218.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.218.252:39503
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.28.218.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4247
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:50:14.866818 4247 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:50:14.867079 4247 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:50:14.872437 4256 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:50:14.872591 4247 server_base.cc:1034] running on GCE node
W20260806 22:50:14.872479 4255 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:14.872700 4258 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:50:14.872956 4247 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:50:14.873252 4247 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:50:14.874421 4247 hybrid_clock.cc:648] HybridClock initialized: now 1786056614874393 us; error 48 us; skew 500 ppm
Aug 06 22:50:14 dist-test-slave-1xrh krb5kdc[3512](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056614, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.218.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:50:14.876953 4247 init.cc:377] Logged in from keytab as oryx/127.28.218.252@KRBTEST.COM (short username oryx)
I20260806 22:50:14.877473 4247 webserver.cc:460] Webserver started at http://127.28.218.252:41413/ using document root <none> and password file <none>
I20260806 22:50:14.877751 4247 fs_manager.cc:359] Metadata directory not provided
I20260806 22:50:14.877816 4247 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:50:14.877954 4247 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:50:14.879222 4247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/master-2/data/instance:
uuid: "ed70ecfd24614dbc990e056d21844d64"
format_stamp: "Formatted at 2026-08-06 22:50:14 on dist-test-slave-1xrh"
server_key: "d2e6b584119fc84127b751701b44ddb9"
server_key_iv: "b2e2ff489cf66c577dadcce3ec8d0ecb"
server_key_version: "encryptionkey@0"
I20260806 22:50:14.879613 4247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/master-2/wal/instance:
uuid: "ed70ecfd24614dbc990e056d21844d64"
format_stamp: "Formatted at 2026-08-06 22:50:14 on dist-test-slave-1xrh"
server_key: "d2e6b584119fc84127b751701b44ddb9"
server_key_iv: "b2e2ff489cf66c577dadcce3ec8d0ecb"
server_key_version: "encryptionkey@0"
I20260806 22:50:14.881014 4247 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 22:50:14.881698 4266 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:14.881851 4247 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:50:14.881953 4247 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/master-2/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/master-2/wal
uuid: "ed70ecfd24614dbc990e056d21844d64"
format_stamp: "Formatted at 2026-08-06 22:50:14 on dist-test-slave-1xrh"
server_key: "d2e6b584119fc84127b751701b44ddb9"
server_key_iv: "b2e2ff489cf66c577dadcce3ec8d0ecb"
server_key_version: "encryptionkey@0"
I20260806 22:50:14.882022 4247 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:50:14.892298 4247 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:50:14.894138 4247 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:50:14.894296 4247 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:50:14.898430 4247 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.252:39503
I20260806 22:50:14.898495 4323 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.252:39503 every 8 connection(s)
I20260806 22:50:14.898860 4247 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/master-2/data/info.pb
I20260806 22:50:14.899885 4324 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:50:14.900584 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 4247
I20260806 22:50:14.901922 4324 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.218.254" port: 44755 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 22:50:14 dist-test-slave-1xrh krb5kdc[3512](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056614, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.218.252@KRBTEST.COM for oryx/127.28.218.254@KRBTEST.COM
Aug 06 22:50:14 dist-test-slave-1xrh krb5kdc[3512](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056613, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.28.218.252@KRBTEST.COM
I20260806 22:50:14.911226 3620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:14.902873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51828 (local address 127.28.218.254:44755)
0806 22:50:14.902939 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:14.902941 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:14.903111 (+ 170us) server_negotiation.cc:407] Connection header received
0806 22:50:14.903224 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:14.903224 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:14.903237 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:14.903300 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:14.903743 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:14.904438 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:14.905320 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:14.905520 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:14.905591 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:14.907771 (+ 2180us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:50:14.907772 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:50:14.907772 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:50:14.907780 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:50:14.909609 (+ 1829us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:14.910060 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:14.910061 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:14.910062 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:14.910126 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:14.910417 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:14.910418 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:14.910418 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:14.910548 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:50:14.910621 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:14.910968 (+ 347us) server_negotiation.cc:299] Negotiation successful
0806 22:50:14.911053 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 22:50:14.911444 4328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:14.902677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:44755 (local address 127.0.0.1:51828)
0806 22:50:14.903028 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:14.903059 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:14.903106 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:14.903322 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:14.903324 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:14.903572 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:14.903659 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:14.903663 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:14.904532 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:14.904533 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:14.905234 (+ 701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:14.905236 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:14.905641 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:14.905642 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:14.905713 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:14.906190 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:50:14.906197 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:14.907684 (+ 1487us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:50:14.909691 (+ 2007us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:14.909697 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:14.909698 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:14.909987 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:14.910197 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:14.910198 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:14.910198 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:14.910347 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:14.910617 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:50:14.910623 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:50:14.910792 (+ 169us) client_negotiation.cc:769] Sending connection context
0806 22:50:14.910891 (+ 99us) client_negotiation.cc:240] Negotiation successful
0806 22:50:14.911037 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":327,"mutex_wait_us":2,"thread_start_us":69,"threads_started":1}
I20260806 22:50:14.911621 4327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:14.901461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45946 (local address 127.28.218.252:39503)
0806 22:50:14.901640 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:14.901648 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:14.901673 (+ 25us) server_negotiation.cc:407] Connection header received
0806 22:50:14.902097 (+ 424us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:14.902104 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:14.902150 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:14.902259 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:14.902999 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:14.904244 (+ 1245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:14.904964 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:14.905365 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:14.905516 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:14.908328 (+ 2812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:50:14.908336 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:50:14.908336 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:50:14.908344 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:50:14.910178 (+ 1834us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:14.910552 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:14.910555 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:14.910556 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:14.910613 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:14.910805 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:14.910806 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:14.910806 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:14.910965 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:50:14.911103 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:14.911239 (+ 136us) server_negotiation.cc:299] Negotiation successful
0806 22:50:14.911516 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":22,"server-negotiator.queue_time_us":138,"thread_start_us":92,"threads_started":1}
I20260806 22:50:14.911990 4324 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.218.253" port: 39243 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 22:50:14 dist-test-slave-1xrh krb5kdc[3512](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056614, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.218.252@KRBTEST.COM for oryx/127.28.218.253@KRBTEST.COM
I20260806 22:50:14.918151 4328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:14.912266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.253:39243 (local address 127.0.0.1:46418)
0806 22:50:14.912322 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:14.912333 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:14.912388 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:14.912487 (+ 99us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:14.912488 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:14.912606 (+ 118us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:14.912647 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:14.912648 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:14.913233 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:14.913234 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:14.913740 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:14.913742 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:14.914120 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:14.914121 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:14.914167 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:14.914528 (+ 361us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:50:14.914531 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:14.915707 (+ 1176us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:50:14.917291 (+ 1584us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:14.917292 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:14.917293 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:14.917456 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:14.917655 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:14.917656 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:14.917657 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:14.917718 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:14.917927 (+ 209us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:50:14.917927 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:50:14.917962 (+ 35us) client_negotiation.cc:769] Sending connection context
0806 22:50:14.918015 (+ 53us) client_negotiation.cc:240] Negotiation successful
0806 22:50:14.918093 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260806 22:50:14.918154 4230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:14.912348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46418 (local address 127.28.218.253:39243)
0806 22:50:14.912397 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:14.912399 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:14.912409 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:50:14.912434 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:14.912434 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:14.912443 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:14.912485 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:14.912704 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:14.913175 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:14.913806 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:14.914046 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:14.914106 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:14.915783 (+ 1677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:50:14.915784 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:50:14.915784 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:50:14.915790 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:50:14.917214 (+ 1424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:14.917529 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:14.917529 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:14.917530 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:14.917577 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:14.917781 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:14.917781 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:14.917782 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:14.917875 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:50:14.917923 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:14.918030 (+ 107us) server_negotiation.cc:299] Negotiation successful
0806 22:50:14.918096 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260806 22:50:14.918615 4324 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.218.252" port: 39503 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 22:50:14 dist-test-slave-1xrh krb5kdc[3512](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056614, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.218.252@KRBTEST.COM for oryx/127.28.218.252@KRBTEST.COM
I20260806 22:50:14.924762 4328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:14.918855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.252:39503 (local address 127.0.0.1:45960)
0806 22:50:14.918904 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:14.918915 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:14.918963 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:14.919087 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:14.919088 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:14.919239 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:14.919263 (+ 24us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:14.919264 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:14.919887 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:14.919888 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:14.920425 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:14.920427 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:14.920783 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:14.920784 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:14.920839 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:14.921226 (+ 387us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:50:14.921232 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:14.922402 (+ 1170us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:50:14.923885 (+ 1483us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:14.923886 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:14.923886 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:14.924093 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:14.924258 (+ 165us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:14.924259 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:14.924259 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:14.924308 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:14.924521 (+ 213us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:50:14.924521 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:50:14.924559 (+ 38us) client_negotiation.cc:769] Sending connection context
0806 22:50:14.924625 (+ 66us) client_negotiation.cc:240] Negotiation successful
0806 22:50:14.924703 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260806 22:50:14.924824 4327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:14.918905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45960 (local address 127.28.218.252:39503)
0806 22:50:14.918939 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:14.918941 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:14.918948 (+ 7us) server_negotiation.cc:407] Connection header received
0806 22:50:14.919019 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:14.919020 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:14.919028 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:14.919083 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:14.919349 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:14.919798 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:14.920511 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:14.920713 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:14.920774 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:14.922478 (+ 1704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:50:14.922479 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:50:14.922480 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:50:14.922485 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:50:14.923812 (+ 1327us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:14.924158 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:14.924159 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:14.924159 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:14.924206 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:14.924355 (+ 149us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:14.924356 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:14.924356 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:14.924445 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:50:14.924494 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:14.924643 (+ 149us) server_negotiation.cc:299] Negotiation successful
0806 22:50:14.924746 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":43,"server-negotiator.queue_time_us":19}
I20260806 22:50:14.926147 4324 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed70ecfd24614dbc990e056d21844d64: Bootstrap starting.
I20260806 22:50:14.927027 4324 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ed70ecfd24614dbc990e056d21844d64: Neither blocks nor log segments found. Creating new log.
I20260806 22:50:14.927356 4324 log.cc:824] T 00000000000000000000000000000000 P ed70ecfd24614dbc990e056d21844d64: Log is configured to *not* fsync() on all Append() calls
I20260806 22:50:14.928242 4324 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed70ecfd24614dbc990e056d21844d64: No bootstrap required, opened a new log
I20260806 22:50:14.929703 4324 raft_consensus.cc:348] T 00000000000000000000000000000000 P ed70ecfd24614dbc990e056d21844d64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20186b499bd34a7d9d497ba785db0fc9" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 44755 } } peers { permanent_uuid: "eb135056735a446dbff018e4072cab0c" member_type: VOTER last_known_addr { host: "127.28.218.253" port: 39243 } } peers { permanent_uuid: "ed70ecfd24614dbc990e056d21844d64" member_type: VOTER last_known_addr { host: "127.28.218.252" port: 39503 } }
I20260806 22:50:14.929847 4324 raft_consensus.cc:374] T 00000000000000000000000000000000 P ed70ecfd24614dbc990e056d21844d64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:50:14.929884 4324 raft_consensus.cc:729] T 00000000000000000000000000000000 P ed70ecfd24614dbc990e056d21844d64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed70ecfd24614dbc990e056d21844d64, State: Initialized, Role: FOLLOWER
I20260806 22:50:14.930003 4324 consensus_queue.cc:260] T 00000000000000000000000000000000 P ed70ecfd24614dbc990e056d21844d64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20186b499bd34a7d9d497ba785db0fc9" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 44755 } } peers { permanent_uuid: "eb135056735a446dbff018e4072cab0c" member_type: VOTER last_known_addr { host: "127.28.218.253" port: 39243 } } peers { permanent_uuid: "ed70ecfd24614dbc990e056d21844d64" member_type: VOTER last_known_addr { host: "127.28.218.252" port: 39503 } }
I20260806 22:50:14.930263 4330 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed70ecfd24614dbc990e056d21844d64 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20186b499bd34a7d9d497ba785db0fc9" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 44755 } } peers { permanent_uuid: "eb135056735a446dbff018e4072cab0c" member_type: VOTER last_known_addr { host: "127.28.218.253" port: 39243 } } peers { permanent_uuid: "ed70ecfd24614dbc990e056d21844d64" member_type: VOTER last_known_addr { host: "127.28.218.252" port: 39503 } } }
I20260806 22:50:14.930305 4324 sys_catalog.cc:564] T 00000000000000000000000000000000 P ed70ecfd24614dbc990e056d21844d64 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:50:14.930341 4330 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed70ecfd24614dbc990e056d21844d64 [sys.catalog]: This master's current role is: FOLLOWER
W20260806 22:50:14.932083 4341 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ed70ecfd24614dbc990e056d21844d64: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 22:50:14.932142 4341 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 22:50:14.934348 4247 master_runner.cc:428] Detected that this master ed70ecfd24614dbc990e056d21844d64 is joining an existing cluster
I20260806 22:50:14.934396 4247 master_runner.cc:432] Initiating AddMaster RPC to add 127.28.218.252:39503
I20260806 22:50:14.942968 4327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:14.935780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45964 (local address 127.28.218.252:39503)
0806 22:50:14.935817 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:14.935819 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:14.935980 (+ 161us) server_negotiation.cc:407] Connection header received
0806 22:50:14.936036 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:14.936037 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:14.936046 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:14.936103 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:14.936506 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:14.937180 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:14.938301 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:14.938481 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:14.938581 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:14.939742 (+ 1161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:50:14.939743 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:50:14.939744 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:50:14.939750 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:50:14.941669 (+ 1919us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:14.942033 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:14.942034 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:14.942035 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:14.942088 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:14.942308 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:14.942309 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:14.942310 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:14.942440 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:50:14.942484 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:14.942679 (+ 195us) server_negotiation.cc:299] Negotiation successful
0806 22:50:14.942862 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 22:50:14.943248 4348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:14.935740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.252:39503 (local address 127.0.0.1:45964)
0806 22:50:14.935908 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:14.935918 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:14.935979 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:14.936108 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:14.936109 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:14.936365 (+ 256us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:14.936428 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:14.936430 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:14.937274 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:14.937275 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:14.938214 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:14.938216 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:14.938616 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:14.938617 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:14.938673 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:14.939204 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:50:14.939209 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:14.939652 (+ 443us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:50:14.941756 (+ 2104us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:14.941756 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:14.941757 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:14.941946 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:14.942156 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:14.942157 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:14.942157 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:14.942240 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:14.942526 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:50:14.942527 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:50:14.942563 (+ 36us) client_negotiation.cc:769] Sending connection context
0806 22:50:14.943051 (+ 488us) client_negotiation.cc:240] Negotiation successful
0806 22:50:14.943161 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":128,"threads_started":1}
I20260806 22:50:14.943883 4347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:14.935494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.253:39243 (local address 127.0.0.1:46420)
0806 22:50:14.935699 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:14.935711 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:14.935802 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:14.936009 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:14.936010 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:14.936249 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:14.936311 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:14.936313 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:14.938316 (+ 2003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:14.938317 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:14.939115 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:14.939119 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:14.939658 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:14.939659 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:14.939724 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:14.940219 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:50:14.940225 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:14.940727 (+ 502us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:50:14.942551 (+ 1824us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:14.942551 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:14.942552 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:14.942839 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:14.943069 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:14.943069 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:14.943070 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:14.943160 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:14.943458 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:50:14.943459 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:50:14.943511 (+ 52us) client_negotiation.cc:769] Sending connection context
0806 22:50:14.943657 (+ 146us) client_negotiation.cc:240] Negotiation successful
0806 22:50:14.943758 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":103,"threads_started":1}
I20260806 22:50:14.943988 4230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:14.935561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46420 (local address 127.28.218.253:39243)
0806 22:50:14.935630 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:14.935632 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:14.935760 (+ 128us) server_negotiation.cc:407] Connection header received
0806 22:50:14.935884 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:14.935885 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:14.935896 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:14.935925 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:14.936387 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:14.937182 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:14.939248 (+ 2066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:14.939562 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:14.939654 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:14.940853 (+ 1199us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:50:14.940855 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:50:14.940855 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:50:14.940862 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:50:14.942457 (+ 1595us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:14.942934 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:14.942936 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:14.942936 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:14.942993 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:14.943238 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:14.943239 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:14.943239 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:14.943365 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:50:14.943409 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:14.943729 (+ 320us) server_negotiation.cc:299] Negotiation successful
0806 22:50:14.943852 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 22:50:14.952800 4347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:14.947319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:44755 (local address 127.0.0.1:51844)
0806 22:50:14.947462 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:14.947473 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:14.947521 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:14.947669 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:14.947669 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:14.947825 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:14.947887 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:14.947889 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:14.948576 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:14.948577 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:14.949168 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:14.949170 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:14.949524 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:14.949525 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:14.949594 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:14.950016 (+ 422us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:50:14.950020 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:14.950280 (+ 260us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:50:14.951818 (+ 1538us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:14.951819 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:14.951820 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:14.951990 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:14.952193 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:14.952194 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:14.952195 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:14.952253 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:14.952489 (+ 236us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:50:14.952490 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:50:14.952531 (+ 41us) client_negotiation.cc:769] Sending connection context
0806 22:50:14.952613 (+ 82us) client_negotiation.cc:240] Negotiation successful
0806 22:50:14.952709 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113}
I20260806 22:50:14.952800 3620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:14.947397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51844 (local address 127.28.218.254:44755)
0806 22:50:14.947457 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:14.947459 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:14.947507 (+ 48us) server_negotiation.cc:407] Connection header received
0806 22:50:14.947584 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:14.947585 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:14.947594 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:14.947664 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:14.947971 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:14.948494 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:14.949257 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:14.949449 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:14.949513 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:14.950383 (+ 870us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:50:14.950384 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:50:14.950384 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:50:14.950391 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:50:14.951727 (+ 1336us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:14.952073 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:14.952074 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:14.952074 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:14.952130 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:14.952319 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:14.952320 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:14.952320 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:14.952424 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:50:14.952498 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:14.952636 (+ 138us) server_negotiation.cc:299] Negotiation successful
0806 22:50:14.952709 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
Aug 06 22:50:14 dist-test-slave-1xrh krb5kdc[3512](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056613, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.218.254@KRBTEST.COM for oryx/127.28.218.252@KRBTEST.COM
I20260806 22:50:14.962819 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 3626
I20260806 22:50:14.968848 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.218.193:35729
--local_ip_for_outbound_sockets=127.28.218.193
--tserver_master_addrs=127.28.218.254:44755
--webserver_port=42159
--webserver_interface=127.28.218.193
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/krb5kdc/oryx_127.28.218.193.keytab
--principal=oryx/127.28.218.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.218.212:37527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.28.218.254:44755,127.28.218.253:39243
--tserver_master_addrs=127.28.218.254:44755,127.28.218.253:39243,127.28.218.252:39503 with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:50:14.982424 4327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:14.954875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45966 (local address 127.28.218.252:39503)
0806 22:50:14.954931 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:14.954933 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:14.954940 (+ 7us) server_negotiation.cc:407] Connection header received
0806 22:50:14.954969 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:14.954970 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:14.954984 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:14.955039 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:14.955354 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:14.955790 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:14.956443 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:14.956643 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:14.956708 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:14.958456 (+ 1748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:50:14.958457 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:50:14.958457 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:50:14.958463 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:50:14.981084 (+ 22621us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:14.981576 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:14.981577 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:14.981577 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:14.981645 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:14.981851 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:14.981851 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:14.981852 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:14.982010 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:50:14.982080 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:14.982239 (+ 159us) server_negotiation.cc:299] Negotiation successful
0806 22:50:14.982325 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 22:50:14.982458 4148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:14.954700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.252:39503 (local address 127.0.0.1:45966)
0806 22:50:14.954749 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:14.954760 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:14.954817 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:14.955055 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:14.955056 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:14.955240 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:14.955276 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:14.955277 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:14.955863 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:14.955864 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:14.956363 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:14.956364 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:14.956720 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:14.956721 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:14.956786 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:14.957165 (+ 379us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:50:14.957170 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:14.958359 (+ 1189us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:50:14.981203 (+ 22844us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:14.981205 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:14.981206 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:14.981483 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:14.981717 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:14.981718 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:14.981718 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:14.981788 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:14.982077 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:50:14.982078 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:50:14.982133 (+ 55us) client_negotiation.cc:769] Sending connection context
0806 22:50:14.982203 (+ 70us) client_negotiation.cc:240] Negotiation successful
0806 22:50:14.982327 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260806 22:50:14.983474 3557 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.28.218.252:39503: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "ed70ecfd24614dbc990e056d21844d64"
member_type: NON_VOTER
last_known_addr {
host: "127.28.218.252"
port: 39503
}
attrs {
promote: true
}
}
dest_uuid: "20186b499bd34a7d9d497ba785db0fc9"
cas_config_opid_index: 5
I20260806 22:50:14.983742 3557 consensus_queue.cc:237] T 00000000000000000000000000000000 P 20186b499bd34a7d9d497ba785db0fc9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "20186b499bd34a7d9d497ba785db0fc9" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 44755 } } peers { permanent_uuid: "eb135056735a446dbff018e4072cab0c" member_type: NON_VOTER last_known_addr { host: "127.28.218.253" port: 39243 } attrs { promote: true } } peers { permanent_uuid: "ed70ecfd24614dbc990e056d21844d64" member_type: NON_VOTER last_known_addr { host: "127.28.218.252" port: 39503 } attrs { promote: true } }
I20260806 22:50:14.984176 4149 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 20186b499bd34a7d9d497ba785db0fc9 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER ed70ecfd24614dbc990e056d21844d64 (127.28.218.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "20186b499bd34a7d9d497ba785db0fc9" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 44755 } } peers { permanent_uuid: "eb135056735a446dbff018e4072cab0c" member_type: NON_VOTER last_known_addr { host: "127.28.218.253" port: 39243 } attrs { promote: true } } peers { permanent_uuid: "ed70ecfd24614dbc990e056d21844d64" member_type: NON_VOTER last_known_addr { host: "127.28.218.252" port: 39503 } attrs { promote: true } } }
I20260806 22:50:14.984617 4295 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ed70ecfd24614dbc990e056d21844d64 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:50:14.985164 4149 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.28.218.252:39503
I20260806 22:50:14.985245 4149 sys_catalog.cc:455] T 00000000000000000000000000000000 P 20186b499bd34a7d9d497ba785db0fc9 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "20186b499bd34a7d9d497ba785db0fc9" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "20186b499bd34a7d9d497ba785db0fc9" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 44755 } } peers { permanent_uuid: "eb135056735a446dbff018e4072cab0c" member_type: NON_VOTER last_known_addr { host: "127.28.218.253" port: 39243 } attrs { promote: true } } peers { permanent_uuid: "ed70ecfd24614dbc990e056d21844d64" member_type: NON_VOTER last_known_addr { host: "127.28.218.252" port: 39503 } attrs { promote: true } } }
I20260806 22:50:14.985342 4149 sys_catalog.cc:458] T 00000000000000000000000000000000 P 20186b499bd34a7d9d497ba785db0fc9 [sys.catalog]: This master's current role is: LEADER
I20260806 22:50:14.986047 4295 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ed70ecfd24614dbc990e056d21844d64 [term 1 FOLLOWER]: Refusing update from remote peer 20186b499bd34a7d9d497ba785db0fc9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260806 22:50:14.987102 4150 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 20186b499bd34a7d9d497ba785db0fc9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed70ecfd24614dbc990e056d21844d64" member_type: NON_VOTER last_known_addr { host: "127.28.218.252" port: 39503 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:50:14.988526 4295 raft_consensus.cc:858] T 00000000000000000000000000000000 P ed70ecfd24614dbc990e056d21844d64 [term 1 NON_PARTICIPANT]: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20186b499bd34a7d9d497ba785db0fc9" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 44755 } } peers { permanent_uuid: "eb135056735a446dbff018e4072cab0c" member_type: VOTER last_known_addr { host: "127.28.218.253" port: 39243 } } peers { permanent_uuid: "ed70ecfd24614dbc990e056d21844d64" member_type: VOTER last_known_addr { host: "127.28.218.252" port: 39503 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "20186b499bd34a7d9d497ba785db0fc9" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 44755 } } peers { permanent_uuid: "eb135056735a446dbff018e4072cab0c" member_type: NON_VOTER last_known_addr { host: "127.28.218.253" port: 39243 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "20186b499bd34a7d9d497ba785db0fc9" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 44755 } } peers { permanent_uuid: "eb135056735a446dbff018e4072cab0c" member_type: NON_VOTER last_known_addr { host: "127.28.218.253" port: 39243 } attrs { promote: true } } peers { permanent_uuid: "ed70ecfd24614dbc990e056d21844d64" member_type: NON_VOTER last_known_addr { host: "127.28.218.252" port: 39503 } attrs { promote: true } }
W20260806 22:50:14.988606 4295 raft_consensus.cc:1565] T 00000000000000000000000000000000 P ed70ecfd24614dbc990e056d21844d64 [term 1 NON_PARTICIPANT]: Could not prepare op 'term: 1 index: 6' and following 0 ops. Status for this op: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20186b499bd34a7d9d497ba785db0fc9" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 44755 } } peers { permanent_uuid: "eb135056735a446dbff018e4072cab0c" member_type: VOTER last_known_addr { host: "127.28.218.253" port: 39243 } } peers { permanent_uuid: "ed70ecfd24614dbc990e056d21844d64" member_type: VOTER last_known_addr { host: "127.28.218.252" port: 39503 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "20186b499bd34a7d9d497ba785db0fc9" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 44755 } } peers { permanent_uuid: "eb135056735a446dbff018e4072cab0c" member_type: NON_VOTER last_known_addr { host: "127.28.218.253" port: 39243 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "20186b499bd34a7d9d497ba785db0fc9" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 44755 } } peers { permanent_uuid: "eb135056735a446dbff018e4072cab0c" member_type: NON_VOTER last_known_addr { host: "127.28.218.253" port: 39243 } attrs { promote: true } } peers { permanent_uuid: "ed70ecfd24614dbc990e056d21844d64" member_type: NON_VOTER last_known_addr { host: "127.28.218.252" port: 39503 } attrs { promote: true } }
I20260806 22:50:14.988938 4247 master.cc:537] Master@127.28.218.252:39503 shutting down...
I20260806 22:50:14.989105 4295 raft_consensus.cc:2932] T 00000000000000000000000000000000 P ed70ecfd24614dbc990e056d21844d64 [term 1 NON_PARTICIPANT]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER eb135056735a446dbff018e4072cab0c (127.28.218.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "20186b499bd34a7d9d497ba785db0fc9" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 44755 } } peers { permanent_uuid: "eb135056735a446dbff018e4072cab0c" member_type: NON_VOTER last_known_addr { host: "127.28.218.253" port: 39243 } attrs { promote: true } } }
I20260806 22:50:14.990127 4330 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed70ecfd24614dbc990e056d21844d64 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 20186b499bd34a7d9d497ba785db0fc9. Latest consensus state: current_term: 1 leader_uuid: "20186b499bd34a7d9d497ba785db0fc9" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "20186b499bd34a7d9d497ba785db0fc9" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 44755 } } peers { permanent_uuid: "eb135056735a446dbff018e4072cab0c" member_type: NON_VOTER last_known_addr { host: "127.28.218.253" port: 39243 } attrs { promote: true } } }
I20260806 22:50:14.990231 4330 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed70ecfd24614dbc990e056d21844d64 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260806 22:50:14.990288 4330 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed70ecfd24614dbc990e056d21844d64 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "20186b499bd34a7d9d497ba785db0fc9" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "20186b499bd34a7d9d497ba785db0fc9" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 44755 } } peers { permanent_uuid: "eb135056735a446dbff018e4072cab0c" member_type: NON_VOTER last_known_addr { host: "127.28.218.253" port: 39243 } attrs { promote: true } } }
I20260806 22:50:14.990355 4330 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed70ecfd24614dbc990e056d21844d64 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260806 22:50:14.990396 4330 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed70ecfd24614dbc990e056d21844d64 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "20186b499bd34a7d9d497ba785db0fc9" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "20186b499bd34a7d9d497ba785db0fc9" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 44755 } } peers { permanent_uuid: "eb135056735a446dbff018e4072cab0c" member_type: NON_VOTER last_known_addr { host: "127.28.218.253" port: 39243 } attrs { promote: true } } }
I20260806 22:50:14.990442 4330 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed70ecfd24614dbc990e056d21844d64 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260806 22:50:14.991902 4148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:14.984331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.253:39243 (local address 127.0.0.1:46424)
0806 22:50:14.985843 (+ 1512us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:14.985854 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:14.985891 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:14.986107 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:14.986108 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:14.986109 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:50:14.986155 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:14.986157 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:14.987362 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:14.987363 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:14.988555 (+ 1192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:14.988559 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:14.991588 (+ 3029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:14.991589 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:14.991668 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:14.991700 (+ 32us) client_negotiation.cc:769] Sending connection context
0806 22:50:14.991766 (+ 66us) client_negotiation.cc:240] Negotiation successful
0806 22:50:14.991814 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1497}
I20260806 22:50:14.991918 4230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:14.985614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46424 (local address 127.28.218.253:39243)
0806 22:50:14.985663 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:14.985665 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:14.985942 (+ 277us) server_negotiation.cc:407] Connection header received
0806 22:50:14.985972 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:14.985973 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:14.985996 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:14.986050 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:50:14.986219 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:14.987010 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:14.990506 (+ 3496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:14.991152 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:14.991232 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:14.991303 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:14.991775 (+ 472us) server_negotiation.cc:299] Negotiation successful
0806 22:50:14.991831 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 22:50:14.992178 4200 raft_consensus.cc:1261] T 00000000000000000000000000000000 P eb135056735a446dbff018e4072cab0c [term 1 LEARNER]: Refusing update from remote peer 20186b499bd34a7d9d497ba785db0fc9: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260806 22:50:14.992385 4150 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 20186b499bd34a7d9d497ba785db0fc9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb135056735a446dbff018e4072cab0c" member_type: NON_VOTER last_known_addr { host: "127.28.218.253" port: 39243 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260806 22:50:14.992961 4200 raft_consensus.cc:2932] T 00000000000000000000000000000000 P eb135056735a446dbff018e4072cab0c [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER ed70ecfd24614dbc990e056d21844d64 (127.28.218.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "20186b499bd34a7d9d497ba785db0fc9" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 44755 } } peers { permanent_uuid: "eb135056735a446dbff018e4072cab0c" member_type: NON_VOTER last_known_addr { host: "127.28.218.253" port: 39243 } attrs { promote: true } } peers { permanent_uuid: "ed70ecfd24614dbc990e056d21844d64" member_type: NON_VOTER last_known_addr { host: "127.28.218.252" port: 39503 } attrs { promote: true } } }
I20260806 22:50:14.994086 4355 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb135056735a446dbff018e4072cab0c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 20186b499bd34a7d9d497ba785db0fc9. Latest consensus state: current_term: 1 leader_uuid: "20186b499bd34a7d9d497ba785db0fc9" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "20186b499bd34a7d9d497ba785db0fc9" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 44755 } } peers { permanent_uuid: "eb135056735a446dbff018e4072cab0c" member_type: NON_VOTER last_known_addr { host: "127.28.218.253" port: 39243 } attrs { promote: true } } peers { permanent_uuid: "ed70ecfd24614dbc990e056d21844d64" member_type: NON_VOTER last_known_addr { host: "127.28.218.252" port: 39503 } attrs { promote: true } } }
I20260806 22:50:14.994133 4232 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb135056735a446dbff018e4072cab0c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "20186b499bd34a7d9d497ba785db0fc9" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "20186b499bd34a7d9d497ba785db0fc9" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 44755 } } peers { permanent_uuid: "eb135056735a446dbff018e4072cab0c" member_type: NON_VOTER last_known_addr { host: "127.28.218.253" port: 39243 } attrs { promote: true } } peers { permanent_uuid: "ed70ecfd24614dbc990e056d21844d64" member_type: NON_VOTER last_known_addr { host: "127.28.218.252" port: 39503 } attrs { promote: true } } }
I20260806 22:50:14.994248 4232 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb135056735a446dbff018e4072cab0c [sys.catalog]: This master's current role is: LEARNER
I20260806 22:50:14.994335 4355 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb135056735a446dbff018e4072cab0c [sys.catalog]: This master's current role is: LEARNER
I20260806 22:50:14.994396 4150 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 20186b499bd34a7d9d497ba785db0fc9: attempting to promote NON_VOTER eb135056735a446dbff018e4072cab0c to VOTER
I20260806 22:50:14.994587 4150 consensus_queue.cc:237] T 00000000000000000000000000000000 P 20186b499bd34a7d9d497ba785db0fc9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "20186b499bd34a7d9d497ba785db0fc9" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 44755 } } peers { permanent_uuid: "eb135056735a446dbff018e4072cab0c" member_type: VOTER last_known_addr { host: "127.28.218.253" port: 39243 } attrs { promote: false } } peers { permanent_uuid: "ed70ecfd24614dbc990e056d21844d64" member_type: NON_VOTER last_known_addr { host: "127.28.218.252" port: 39503 } attrs { promote: true } }
W20260806 22:50:14.995400 3543 consensus_peers.cc:487] T 00000000000000000000000000000000 P 20186b499bd34a7d9d497ba785db0fc9 -> Peer ed70ecfd24614dbc990e056d21844d64 (127.28.218.252:39503): Couldn't send request to peer ed70ecfd24614dbc990e056d21844d64. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260806 22:50:14.995913 4200 raft_consensus.cc:1261] T 00000000000000000000000000000000 P eb135056735a446dbff018e4072cab0c [term 1 LEARNER]: Refusing update from remote peer 20186b499bd34a7d9d497ba785db0fc9: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260806 22:50:14.996172 4350 mvcc.cc:205] Tried to move back new op lower bound from 7315687890580697088 to 7315687890446454784. Current Snapshot: MvccSnapshot[applied={T|T < 7315687890457321472}]
I20260806 22:50:14.996393 4247 raft_consensus.cc:2229] T 00000000000000000000000000000000 P ed70ecfd24614dbc990e056d21844d64 [term 1 NON_PARTICIPANT]: Raft consensus shutting down.
I20260806 22:50:14.996490 4247 raft_consensus.cc:2258] T 00000000000000000000000000000000 P ed70ecfd24614dbc990e056d21844d64 [term 1 NON_PARTICIPANT]: Raft consensus is shut down!
I20260806 22:50:14.996529 4247 tablet_replica.cc:330] T 00000000000000000000000000000000 P ed70ecfd24614dbc990e056d21844d64: stopping tablet replica
I20260806 22:50:14.996712 4149 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 20186b499bd34a7d9d497ba785db0fc9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb135056735a446dbff018e4072cab0c" member_type: VOTER last_known_addr { host: "127.28.218.253" port: 39243 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260806 22:50:14.997458 4149 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 20186b499bd34a7d9d497ba785db0fc9 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, eb135056735a446dbff018e4072cab0c (127.28.218.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "20186b499bd34a7d9d497ba785db0fc9" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 44755 } } peers { permanent_uuid: "eb135056735a446dbff018e4072cab0c" member_type: VOTER last_known_addr { host: "127.28.218.253" port: 39243 } attrs { promote: false } } peers { permanent_uuid: "ed70ecfd24614dbc990e056d21844d64" member_type: NON_VOTER last_known_addr { host: "127.28.218.252" port: 39503 } attrs { promote: true } } }
I20260806 22:50:14.997670 4200 raft_consensus.cc:2932] T 00000000000000000000000000000000 P eb135056735a446dbff018e4072cab0c [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, eb135056735a446dbff018e4072cab0c (127.28.218.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "20186b499bd34a7d9d497ba785db0fc9" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 44755 } } peers { permanent_uuid: "eb135056735a446dbff018e4072cab0c" member_type: VOTER last_known_addr { host: "127.28.218.253" port: 39243 } attrs { promote: false } } peers { permanent_uuid: "ed70ecfd24614dbc990e056d21844d64" member_type: NON_VOTER last_known_addr { host: "127.28.218.252" port: 39503 } attrs { promote: true } } }
I20260806 22:50:14.998323 4149 sys_catalog.cc:455] T 00000000000000000000000000000000 P 20186b499bd34a7d9d497ba785db0fc9 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "20186b499bd34a7d9d497ba785db0fc9" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "20186b499bd34a7d9d497ba785db0fc9" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 44755 } } peers { permanent_uuid: "eb135056735a446dbff018e4072cab0c" member_type: VOTER last_known_addr { host: "127.28.218.253" port: 39243 } attrs { promote: false } } peers { permanent_uuid: "ed70ecfd24614dbc990e056d21844d64" member_type: NON_VOTER last_known_addr { host: "127.28.218.252" port: 39503 } attrs { promote: true } } }
I20260806 22:50:14.998391 4149 sys_catalog.cc:458] T 00000000000000000000000000000000 P 20186b499bd34a7d9d497ba785db0fc9 [sys.catalog]: This master's current role is: LEADER
I20260806 22:50:14.998453 4149 sys_catalog.cc:455] T 00000000000000000000000000000000 P 20186b499bd34a7d9d497ba785db0fc9 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "20186b499bd34a7d9d497ba785db0fc9" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "20186b499bd34a7d9d497ba785db0fc9" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 44755 } } peers { permanent_uuid: "eb135056735a446dbff018e4072cab0c" member_type: VOTER last_known_addr { host: "127.28.218.253" port: 39243 } attrs { promote: false } } peers { permanent_uuid: "ed70ecfd24614dbc990e056d21844d64" member_type: NON_VOTER last_known_addr { host: "127.28.218.252" port: 39503 } attrs { promote: true } } }
I20260806 22:50:14.998497 4149 sys_catalog.cc:458] T 00000000000000000000000000000000 P 20186b499bd34a7d9d497ba785db0fc9 [sys.catalog]: This master's current role is: LEADER
I20260806 22:50:14.998548 4355 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb135056735a446dbff018e4072cab0c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "20186b499bd34a7d9d497ba785db0fc9" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "20186b499bd34a7d9d497ba785db0fc9" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 44755 } } peers { permanent_uuid: "eb135056735a446dbff018e4072cab0c" member_type: VOTER last_known_addr { host: "127.28.218.253" port: 39243 } attrs { promote: false } } peers { permanent_uuid: "ed70ecfd24614dbc990e056d21844d64" member_type: NON_VOTER last_known_addr { host: "127.28.218.252" port: 39503 } attrs { promote: true } } }
I20260806 22:50:14.998620 4355 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb135056735a446dbff018e4072cab0c [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:50:15.014837 4247 master.cc:552] Master@127.28.218.252:39503 shutdown complete.
I20260806 22:50:15.014919 4247 master_runner.cc:305] Clearing existing system tablet
I20260806 22:50:15.015081 4247 fs_manager.cc:359] Metadata directory not provided
I20260806 22:50:15.015172 4247 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:50:15.016029 4247 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:50:15.016659 4360 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:15.016824 4247 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:50:15.016906 4247 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:50:15.016935 4247 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/master-2/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/master-2/wal
uuid: "ed70ecfd24614dbc990e056d21844d64"
format_stamp: "Formatted at 2026-08-06 22:50:14 on dist-test-slave-1xrh"
server_key: "d2e6b584119fc84127b751701b44ddb9"
server_key_iv: "b2e2ff489cf66c577dadcce3ec8d0ecb"
server_key_version: "encryptionkey@0"
I20260806 22:50:15.017694 4247 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P ed70ecfd24614dbc990e056d21844d64: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:50:15.019368 4247 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P ed70ecfd24614dbc990e056d21844d64: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260806 22:50:15.019444 4247 log.cc:1196] T 00000000000000000000000000000000 P ed70ecfd24614dbc990e056d21844d64: Deleting WAL directory at /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260806 22:50:15.019701 4247 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P ed70ecfd24614dbc990e056d21844d64: Deleting consensus metadata
I20260806 22:50:15.019838 4247 master_runner.cc:315] Copying system tablet from 127.28.218.254:44755
I20260806 22:50:15.020520 4247 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P ed70ecfd24614dbc990e056d21844d64: tablet copy: Beginning tablet copy session from remote peer at address 127.28.218.254:44755
I20260806 22:50:15.028250 4368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:15.020840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:44755 (local address 127.0.0.1:51860)
0806 22:50:15.021015 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:15.021027 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:15.021116 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:15.021350 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:15.021351 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:15.021561 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:15.021620 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:15.021621 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:15.022562 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:15.022564 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:15.023512 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:15.023515 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:15.024039 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:15.024040 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:15.024103 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:15.024678 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:50:15.024684 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:15.025120 (+ 436us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:50:15.027060 (+ 1940us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:15.027061 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:15.027062 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:15.027321 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:15.027551 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:15.027552 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:15.027553 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:15.027619 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:15.027920 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:50:15.027921 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:50:15.027970 (+ 49us) client_negotiation.cc:769] Sending connection context
0806 22:50:15.028040 (+ 70us) client_negotiation.cc:240] Negotiation successful
0806 22:50:15.028147 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":68,"threads_started":1}
I20260806 22:50:15.028261 3620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:15.020975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51860 (local address 127.28.218.254:44755)
0806 22:50:15.021045 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:15.021047 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:15.021086 (+ 39us) server_negotiation.cc:407] Connection header received
0806 22:50:15.021196 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:15.021197 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:15.021212 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:15.021302 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:15.021714 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:15.022464 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:15.023608 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:15.023948 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:15.024033 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:15.025221 (+ 1188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:50:15.025223 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:50:15.025223 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:50:15.025231 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:50:15.026970 (+ 1739us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:15.027403 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:15.027404 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:15.027404 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:15.027472 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:15.027691 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:15.027692 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:15.027692 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:15.027839 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:50:15.027925 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:15.028083 (+ 158us) server_negotiation.cc:299] Negotiation successful
0806 22:50:15.028168 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 22:50:15.029075 3582 tablet_copy_service.cc:140] P 20186b499bd34a7d9d497ba785db0fc9: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer ed70ecfd24614dbc990e056d21844d64 ({username='oryx', principal='oryx/127.28.218.252@KRBTEST.COM'} at 127.0.0.1:51860)
I20260806 22:50:15.029153 3582 tablet_copy_service.cc:161] P 20186b499bd34a7d9d497ba785db0fc9: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer ed70ecfd24614dbc990e056d21844d64 at {username='oryx', principal='oryx/127.28.218.252@KRBTEST.COM'} at 127.0.0.1:51860: session id = ed70ecfd24614dbc990e056d21844d64-00000000000000000000000000000000
I20260806 22:50:15.029779 3582 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 20186b499bd34a7d9d497ba785db0fc9: Tablet Copy: opened 0 blocks and 1 log segments
I20260806 22:50:15.030342 4247 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260806 22:50:15.031750 4247 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P ed70ecfd24614dbc990e056d21844d64: tablet copy: Starting download of 0 data blocks...
I20260806 22:50:15.031924 4247 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P ed70ecfd24614dbc990e056d21844d64: tablet copy: Starting download of 1 WAL segments...
I20260806 22:50:15.032701 4247 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P ed70ecfd24614dbc990e056d21844d64: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260806 22:50:15.033638 3582 tablet_copy_service.cc:342] P 20186b499bd34a7d9d497ba785db0fc9: Request end of tablet copy session ed70ecfd24614dbc990e056d21844d64-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.28.218.252@KRBTEST.COM'} at 127.0.0.1:51860
I20260806 22:50:15.033694 3582 tablet_copy_service.cc:433] P 20186b499bd34a7d9d497ba785db0fc9: ending tablet copy session ed70ecfd24614dbc990e056d21844d64-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer ed70ecfd24614dbc990e056d21844d64
I20260806 22:50:15.036855 4247 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:50:15.038242 4371 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:15.039983 4370 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:50:15.040112 4247 server_base.cc:1034] running on GCE node
W20260806 22:50:15.043947 4373 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:50:15.044185 4247 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:50:15.044348 4247 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:50:15.045490 4247 hybrid_clock.cc:648] HybridClock initialized: now 1786056615045495 us; error 64 us; skew 500 ppm
Aug 06 22:50:15 dist-test-slave-1xrh krb5kdc[3512](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056615, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.218.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:50:15.047086 4247 init.cc:377] Logged in from keytab as oryx/127.28.218.252@KRBTEST.COM (short username oryx)
I20260806 22:50:15.047403 4247 webserver.cc:460] Webserver started at http://127.28.218.252:45333/ using document root <none> and password file <none>
I20260806 22:50:15.047539 4247 fs_manager.cc:359] Metadata directory not provided
I20260806 22:50:15.047576 4247 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:50:15.048377 4247 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:50:15.048998 4381 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:15.049145 4247 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:50:15.049221 4247 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/master-2/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/master-2/wal
uuid: "ed70ecfd24614dbc990e056d21844d64"
format_stamp: "Formatted at 2026-08-06 22:50:14 on dist-test-slave-1xrh"
server_key: "d2e6b584119fc84127b751701b44ddb9"
server_key_iv: "b2e2ff489cf66c577dadcce3ec8d0ecb"
server_key_version: "encryptionkey@0"
I20260806 22:50:15.049324 4247 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:50:15.064106 4247 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:50:15.066170 4247 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:50:15.069515 4247 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.252:39503
I20260806 22:50:15.069561 4438 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.252:39503 every 8 connection(s)
I20260806 22:50:15.070017 4247 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/master-2/data/info.pb
I20260806 22:50:15.071077 4439 sys_catalog.cc:263] Verifying existing consensus state
I20260806 22:50:15.071620 4439 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed70ecfd24614dbc990e056d21844d64: Bootstrap starting.
I20260806 22:50:15.074126 4439 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed70ecfd24614dbc990e056d21844d64: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 22:50:15.074364 4439 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed70ecfd24614dbc990e056d21844d64: Bootstrap complete.
I20260806 22:50:15.074538 4439 raft_consensus.cc:348] T 00000000000000000000000000000000 P ed70ecfd24614dbc990e056d21844d64 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "20186b499bd34a7d9d497ba785db0fc9" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 44755 } } peers { permanent_uuid: "eb135056735a446dbff018e4072cab0c" member_type: VOTER last_known_addr { host: "127.28.218.253" port: 39243 } attrs { promote: false } } peers { permanent_uuid: "ed70ecfd24614dbc990e056d21844d64" member_type: NON_VOTER last_known_addr { host: "127.28.218.252" port: 39503 } attrs { promote: true } }
I20260806 22:50:15.074617 4439 raft_consensus.cc:729] T 00000000000000000000000000000000 P ed70ecfd24614dbc990e056d21844d64 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ed70ecfd24614dbc990e056d21844d64, State: Initialized, Role: LEARNER
I20260806 22:50:15.074649 4439 consensus_queue.cc:260] T 00000000000000000000000000000000 P ed70ecfd24614dbc990e056d21844d64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "20186b499bd34a7d9d497ba785db0fc9" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 44755 } } peers { permanent_uuid: "eb135056735a446dbff018e4072cab0c" member_type: VOTER last_known_addr { host: "127.28.218.253" port: 39243 } attrs { promote: false } } peers { permanent_uuid: "ed70ecfd24614dbc990e056d21844d64" member_type: NON_VOTER last_known_addr { host: "127.28.218.252" port: 39503 } attrs { promote: true } }
I20260806 22:50:15.074798 4439 sys_catalog.cc:564] T 00000000000000000000000000000000 P ed70ecfd24614dbc990e056d21844d64 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:50:15.074822 4446 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed70ecfd24614dbc990e056d21844d64 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "20186b499bd34a7d9d497ba785db0fc9" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 44755 } } peers { permanent_uuid: "eb135056735a446dbff018e4072cab0c" member_type: VOTER last_known_addr { host: "127.28.218.253" port: 39243 } attrs { promote: false } } peers { permanent_uuid: "ed70ecfd24614dbc990e056d21844d64" member_type: NON_VOTER last_known_addr { host: "127.28.218.252" port: 39503 } attrs { promote: true } } }
I20260806 22:50:15.075057 4446 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed70ecfd24614dbc990e056d21844d64 [sys.catalog]: This master's current role is: LEARNER
W20260806 22:50:15.076308 4349 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:50:15.076565 4349 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:50:15.076610 4349 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260806 22:50:15.076292 4458 catalog_manager.cc:1259] Loaded cluster ID: 49a945fc1aed48f391333d0beb7d3f21
I20260806 22:50:15.076833 4458 catalog_manager.cc:1552] T 00000000000000000000000000000000 P ed70ecfd24614dbc990e056d21844d64: loading cluster ID for follower catalog manager: success
I20260806 22:50:15.078033 4458 catalog_manager.cc:1574] T 00000000000000000000000000000000 P ed70ecfd24614dbc990e056d21844d64: acquiring CA information for follower catalog manager: success
W20260806 22:50:15.078315 4349 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:50:15.078370 4349 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:50:15.078411 4349 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.218.193
I20260806 22:50:15.078516 4458 catalog_manager.cc:1602] T 00000000000000000000000000000000 P ed70ecfd24614dbc990e056d21844d64: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 22:50:15.080123 4349 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:37527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/krb5kdc/oryx_127.28.218.193.keytab
--principal=oryx/127.28.218.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.218.193:35729
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.218.193
--webserver_port=42159
--tserver_master_addrs=127.28.218.254:44755,127.28.218.253:39243,127.28.218.252:39503
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4349
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.218.193
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:50:15.080384 4349 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:50:15.080628 4349 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:50:15.085589 4464 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:15.085613 4461 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:15.085597 4462 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:50:15.086036 4349 server_base.cc:1034] running on GCE node
I20260806 22:50:15.086265 4349 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:50:15.086513 4349 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:50:15.087677 4349 hybrid_clock.cc:648] HybridClock initialized: now 1786056615087660 us; error 34 us; skew 500 ppm
Aug 06 22:50:15 dist-test-slave-1xrh krb5kdc[3512](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056615, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.218.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:50:15.090508 4349 init.cc:377] Logged in from keytab as oryx/127.28.218.193@KRBTEST.COM (short username oryx)
I20260806 22:50:15.091023 4349 webserver.cc:460] Webserver started at http://127.28.218.193:42159/ using document root <none> and password file <none>
I20260806 22:50:15.091292 4349 fs_manager.cc:359] Metadata directory not provided
I20260806 22:50:15.091367 4349 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:50:15.092711 4349 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:50:15.093330 4471 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:15.093488 4349 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:50:15.093570 4349 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/ts-0/wal
uuid: "7785c46d26114840970c8f163edf0196"
format_stamp: "Formatted at 2026-08-06 22:50:14 on dist-test-slave-1xrh"
server_key: "48d61b013e9f1bd16e158253001d95a7"
server_key_iv: "fcebbe89abf853e8145c5b7fd88a99c8"
server_key_version: "encryptionkey@0"
I20260806 22:50:15.093906 4349 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:50:15.112973 4349 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:50:15.115433 4349 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:50:15.115585 4349 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:50:15.115837 4349 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:50:15.116189 4349 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:50:15.116227 4349 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:15.116251 4349 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:50:15.116264 4349 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:15.122622 4349 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.193:35729
I20260806 22:50:15.122709 4594 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.193:35729 every 8 connection(s)
I20260806 22:50:15.123054 4349 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/ts-0/data/info.pb
I20260806 22:50:15.126025 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 4349
I20260806 22:50:15.126186 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 3768
Aug 06 22:50:15 dist-test-slave-1xrh krb5kdc[3512](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056615, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.218.193@KRBTEST.COM for oryx/127.28.218.253@KRBTEST.COM
Aug 06 22:50:15 dist-test-slave-1xrh krb5kdc[3512](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056615, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.218.193@KRBTEST.COM for oryx/127.28.218.252@KRBTEST.COM
I20260806 22:50:15.136224 4230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:15.124260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.193:40425 (local address 127.28.218.253:39243)
0806 22:50:15.124321 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:15.124324 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:15.124406 (+ 82us) server_negotiation.cc:407] Connection header received
0806 22:50:15.125323 (+ 917us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:15.125325 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:15.125336 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:15.125399 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:15.126427 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:15.127112 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:15.128836 (+ 1724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:15.129115 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:15.129154 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:15.132571 (+ 3417us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:50:15.132572 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:50:15.132573 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:50:15.132581 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:50:15.134426 (+ 1845us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:15.134946 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:15.134947 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:15.134948 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:15.135023 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:15.135377 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:15.135378 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:15.135378 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:15.135546 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:50:15.135596 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:15.135984 (+ 388us) server_negotiation.cc:299] Negotiation successful
0806 22:50:15.136076 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260806 22:50:15.136534 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.218.194:45405
--local_ip_for_outbound_sockets=127.28.218.194
--tserver_master_addrs=127.28.218.254:44755
--webserver_port=41751
--webserver_interface=127.28.218.194
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/krb5kdc/oryx_127.28.218.194.keytab
--principal=oryx/127.28.218.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.218.212:37527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.28.218.254:44755,127.28.218.253:39243
--tserver_master_addrs=127.28.218.254:44755,127.28.218.253:39243,127.28.218.252:39503 with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 22:50:15 dist-test-slave-1xrh krb5kdc[3512](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056615, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.218.193@KRBTEST.COM for oryx/127.28.218.254@KRBTEST.COM
I20260806 22:50:15.136677 4599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:15.124180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.253:39243 (local address 127.28.218.193:40425)
0806 22:50:15.124342 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:15.124351 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:15.125157 (+ 806us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:15.125437 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:15.125443 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:15.125975 (+ 532us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:15.126321 (+ 346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:15.126327 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:15.127362 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:15.127363 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:15.128739 (+ 1376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:15.128742 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:15.129235 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:15.129236 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:15.129372 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:15.129921 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:50:15.129940 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:15.132466 (+ 2526us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:50:15.134522 (+ 2056us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:15.134530 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:15.134533 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:15.134853 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:15.135095 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:15.135096 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:15.135096 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:15.135284 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:15.135649 (+ 365us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:50:15.135657 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:50:15.135825 (+ 168us) client_negotiation.cc:769] Sending connection context
0806 22:50:15.136328 (+ 503us) client_negotiation.cc:240] Negotiation successful
0806 22:50:15.136453 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"mutex_wait_us":137,"thread_start_us":64,"threads_started":1}
I20260806 22:50:15.136861 4601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:15.123921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.193:55995 (local address 127.28.218.252:39503)
0806 22:50:15.124156 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:15.124158 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:15.124169 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:50:15.125450 (+ 1281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:15.125451 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:15.125461 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:15.125536 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:15.126438 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:15.127513 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:15.128986 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:15.129413 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:15.129483 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:15.133024 (+ 3541us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:50:15.133025 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:50:15.133026 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:50:15.133033 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:50:15.134722 (+ 1689us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:15.135163 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:15.135164 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:15.135165 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:15.135231 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:15.135445 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:15.135445 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:15.135446 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:15.135600 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:50:15.135669 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:15.136441 (+ 772us) server_negotiation.cc:299] Negotiation successful
0806 22:50:15.136535 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":205,"thread_start_us":55,"threads_started":1}
I20260806 22:50:15.137084 4600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:15.123841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.252:39503 (local address 127.28.218.193:55995)
0806 22:50:15.124036 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:15.124080 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:15.125141 (+ 1061us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:15.125543 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:15.125544 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:15.125973 (+ 429us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:15.126321 (+ 348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:15.126328 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:15.127599 (+ 1271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:15.127600 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:15.128873 (+ 1273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:15.128876 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:15.129548 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:15.129549 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:15.129633 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:15.130086 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:50:15.130091 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:15.132897 (+ 2806us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:50:15.134804 (+ 1907us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:15.134805 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:15.134805 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:15.135042 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:15.135304 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:15.135305 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:15.135305 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:15.135373 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:15.135698 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:50:15.135699 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:50:15.135825 (+ 126us) client_negotiation.cc:769] Sending connection context
0806 22:50:15.135952 (+ 127us) client_negotiation.cc:240] Negotiation successful
0806 22:50:15.136144 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"mutex_wait_us":9,"thread_start_us":123,"threads_started":1}
I20260806 22:50:15.137662 4595 heartbeater.cc:347] Connected to a master server at 127.28.218.252:39503
I20260806 22:50:15.137773 4595 heartbeater.cc:464] Registering TS with master...
I20260806 22:50:15.138005 4595 heartbeater.cc:511] Master 127.28.218.252:39503 requested a full tablet report, sending...
I20260806 22:50:15.139603 4597 heartbeater.cc:347] Connected to a master server at 127.28.218.253:39243
I20260806 22:50:15.139624 4399 ts_manager.cc:194] Registered new tserver with Master: 7785c46d26114840970c8f163edf0196 (127.28.218.193:35729)
I20260806 22:50:15.139659 4597 heartbeater.cc:464] Registering TS with master...
I20260806 22:50:15.139729 4597 heartbeater.cc:511] Master 127.28.218.253:39243 requested a full tablet report, sending...
I20260806 22:50:15.140185 4190 ts_manager.cc:194] Registered new tserver with Master: 7785c46d26114840970c8f163edf0196 (127.28.218.193:35729)
I20260806 22:50:15.140895 3620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:15.124041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.193:43725 (local address 127.28.218.254:44755)
0806 22:50:15.124092 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:15.124094 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:15.124370 (+ 276us) server_negotiation.cc:407] Connection header received
0806 22:50:15.125265 (+ 895us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:15.125266 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:15.125280 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:15.125336 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:15.131610 (+ 6274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:15.132357 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:15.134489 (+ 2132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:15.134854 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:15.134898 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:15.137712 (+ 2814us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:50:15.137713 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:50:15.137714 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:50:15.137723 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:50:15.139490 (+ 1767us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:15.140122 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:15.140124 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:15.140124 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:15.140184 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:15.140394 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:15.140395 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:15.140395 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:15.140530 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:50:15.140596 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:15.140721 (+ 125us) server_negotiation.cc:299] Negotiation successful
0806 22:50:15.140795 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 22:50:15.140929 4602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:15.123841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:44755 (local address 127.28.218.193:43725)
0806 22:50:15.124310 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:15.124320 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:15.125144 (+ 824us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:15.125444 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:15.125444 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:15.125974 (+ 530us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:15.131489 (+ 5515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:15.131493 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:15.132564 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:15.132564 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:15.134363 (+ 1799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:15.134366 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:15.134983 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:15.134984 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:15.135059 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:15.135523 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:50:15.135528 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:15.137594 (+ 2066us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:50:15.139831 (+ 2237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:15.139832 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:15.139832 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:15.140013 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:15.140257 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:15.140258 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:15.140258 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:15.140324 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:15.140586 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:50:15.140587 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:50:15.140626 (+ 39us) client_negotiation.cc:769] Sending connection context
0806 22:50:15.140691 (+ 65us) client_negotiation.cc:240] Negotiation successful
0806 22:50:15.140795 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":438,"mutex_wait_us":29,"thread_start_us":113,"threads_started":1}
I20260806 22:50:15.141270 4596 heartbeater.cc:347] Connected to a master server at 127.28.218.254:44755
I20260806 22:50:15.141330 4596 heartbeater.cc:464] Registering TS with master...
I20260806 22:50:15.141392 4596 heartbeater.cc:511] Master 127.28.218.254:44755 requested a full tablet report, sending...
I20260806 22:50:15.141690 3557 ts_manager.cc:194] Re-registered known tserver with Master: 7785c46d26114840970c8f163edf0196 (127.28.218.193:35729)
I20260806 22:50:15.142056 3557 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.28.218.193@KRBTEST.COM'} at 127.28.218.193:43725
W20260806 22:50:15.222107 4603 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:50:15.222321 4603 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:50:15.222342 4603 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:50:15.223532 4603 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:50:15.223574 4603 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:50:15.223600 4603 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.218.194
I20260806 22:50:15.225117 4603 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:37527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/krb5kdc/oryx_127.28.218.194.keytab
--principal=oryx/127.28.218.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.218.194:45405
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.218.194
--webserver_port=41751
--tserver_master_addrs=127.28.218.254:44755,127.28.218.253:39243,127.28.218.252:39503
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4603
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.218.194
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:50:15.225358 4603 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:50:15.225593 4603 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:50:15.230454 4611 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:15.230448 4609 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:15.230448 4608 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:50:15.230702 4603 server_base.cc:1034] running on GCE node
I20260806 22:50:15.230953 4603 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:50:15.231262 4603 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:50:15.232445 4603 hybrid_clock.cc:648] HybridClock initialized: now 1786056615232446 us; error 40 us; skew 500 ppm
Aug 06 22:50:15 dist-test-slave-1xrh krb5kdc[3512](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056615, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.218.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:50:15.235095 4603 init.cc:377] Logged in from keytab as oryx/127.28.218.194@KRBTEST.COM (short username oryx)
I20260806 22:50:15.235607 4603 webserver.cc:460] Webserver started at http://127.28.218.194:41751/ using document root <none> and password file <none>
I20260806 22:50:15.235850 4603 fs_manager.cc:359] Metadata directory not provided
I20260806 22:50:15.235910 4603 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:50:15.237315 4603 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:50:15.237917 4619 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:15.238080 4603 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:50:15.238165 4603 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/ts-1/wal
uuid: "05b84cde108549378d6584ef921fed23"
format_stamp: "Formatted at 2026-08-06 22:50:14 on dist-test-slave-1xrh"
server_key: "717849e1ee3b8526f5950f6a20e59bbf"
server_key_iv: "041930f4fdf9054828ce2d5e4983d3cd"
server_key_version: "encryptionkey@0"
I20260806 22:50:15.238481 4603 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:50:15.252377 4603 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:50:15.254262 4603 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:50:15.254405 4603 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:50:15.254642 4603 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:50:15.254943 4603 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:50:15.254976 4603 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:15.255012 4603 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:50:15.255038 4603 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:15.260680 4603 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.194:45405
I20260806 22:50:15.260780 4739 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.194:45405 every 8 connection(s)
I20260806 22:50:15.261106 4603 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/ts-1/data/info.pb
I20260806 22:50:15.263365 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 4603
I20260806 22:50:15.263546 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 3907
Aug 06 22:50:15 dist-test-slave-1xrh krb5kdc[3512](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056615, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.218.194@KRBTEST.COM for oryx/127.28.218.253@KRBTEST.COM
Aug 06 22:50:15 dist-test-slave-1xrh krb5kdc[3512](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056615, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.218.194@KRBTEST.COM for oryx/127.28.218.254@KRBTEST.COM
Aug 06 22:50:15 dist-test-slave-1xrh krb5kdc[3512](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056615, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.218.194@KRBTEST.COM for oryx/127.28.218.252@KRBTEST.COM
I20260806 22:50:15.273531 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.218.195:42091
--local_ip_for_outbound_sockets=127.28.218.195
--tserver_master_addrs=127.28.218.254:44755
--webserver_port=46163
--webserver_interface=127.28.218.195
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/krb5kdc/oryx_127.28.218.195.keytab
--principal=oryx/127.28.218.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.218.212:37527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.28.218.254:44755,127.28.218.253:39243
--tserver_master_addrs=127.28.218.254:44755,127.28.218.253:39243,127.28.218.252:39503 with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:50:15.274394 4230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:15.262368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.194:39623 (local address 127.28.218.253:39243)
0806 22:50:15.262415 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:15.262417 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:15.262483 (+ 66us) server_negotiation.cc:407] Connection header received
0806 22:50:15.263745 (+ 1262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:15.263747 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:15.263760 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:15.263794 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:15.264970 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:15.265776 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:15.267510 (+ 1734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:15.267825 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:15.267861 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:15.270817 (+ 2956us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:50:15.270819 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:50:15.270819 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:50:15.270828 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:50:15.272697 (+ 1869us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:15.273230 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:15.273232 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:15.273232 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:15.273295 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:15.273595 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:15.273595 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:15.273596 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:15.273744 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:50:15.273789 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:15.274175 (+ 386us) server_negotiation.cc:299] Negotiation successful
0806 22:50:15.274261 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260806 22:50:15.274762 3620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:15.261851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.194:47699 (local address 127.28.218.254:44755)
0806 22:50:15.261909 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:15.261911 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:15.262359 (+ 448us) server_negotiation.cc:407] Connection header received
0806 22:50:15.263745 (+ 1386us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:15.263746 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:15.263760 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:15.263794 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:15.264969 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:15.265776 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:15.267510 (+ 1734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:15.267812 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:15.267858 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:15.271420 (+ 3562us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:50:15.271422 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:50:15.271422 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:50:15.271431 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:50:15.273596 (+ 2165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:15.274020 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:15.274021 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:15.274022 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:15.274075 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:15.274233 (+ 158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:15.274234 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:15.274234 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:15.274346 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:50:15.274388 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:15.274551 (+ 163us) server_negotiation.cc:299] Negotiation successful
0806 22:50:15.274625 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 22:50:15.274978 4745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:15.262293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.253:39243 (local address 127.28.218.194:39623)
0806 22:50:15.262430 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:15.262439 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:15.263615 (+ 1176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:15.263977 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:15.263978 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:15.264533 (+ 555us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:15.264854 (+ 321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:15.264860 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:15.265925 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:15.265926 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:15.267403 (+ 1477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:15.267405 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:15.267955 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:15.267956 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:15.268101 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:15.268763 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:50:15.268771 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:15.270713 (+ 1942us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:50:15.272793 (+ 2080us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:15.272800 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:15.272803 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:15.273141 (+ 338us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:15.273375 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:15.273375 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:15.273376 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:15.273511 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:15.273839 (+ 328us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:50:15.273846 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:50:15.274030 (+ 184us) client_negotiation.cc:769] Sending connection context
0806 22:50:15.274490 (+ 460us) client_negotiation.cc:240] Negotiation successful
0806 22:50:15.274678 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"mutex_wait_us":159,"thread_start_us":72,"threads_started":1}
I20260806 22:50:15.275357 4744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:15.261795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:44755 (local address 127.28.218.194:47699)
0806 22:50:15.262223 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:15.262267 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:15.263646 (+ 1379us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:15.263977 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:15.263978 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:15.264527 (+ 549us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:15.264854 (+ 327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:15.264860 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:15.265920 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:15.265921 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:15.267402 (+ 1481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:15.267405 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:15.267954 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:15.267955 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:15.268101 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:15.268807 (+ 706us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:50:15.268813 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:15.271308 (+ 2495us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:50:15.273692 (+ 2384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:15.273693 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:15.273693 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:15.273942 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:15.274130 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:15.274130 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:15.274131 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:15.274181 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:15.274413 (+ 232us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:50:15.274414 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:50:15.274450 (+ 36us) client_negotiation.cc:769] Sending connection context
0806 22:50:15.274847 (+ 397us) client_negotiation.cc:240] Negotiation successful
0806 22:50:15.274974 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":392,"mutex_wait_us":162,"thread_start_us":262,"threads_started":1}
I20260806 22:50:15.275508 4742 heartbeater.cc:347] Connected to a master server at 127.28.218.253:39243
I20260806 22:50:15.275605 4742 heartbeater.cc:464] Registering TS with master...
I20260806 22:50:15.275681 4741 heartbeater.cc:347] Connected to a master server at 127.28.218.254:44755
I20260806 22:50:15.275729 4741 heartbeater.cc:464] Registering TS with master...
I20260806 22:50:15.275796 4742 heartbeater.cc:511] Master 127.28.218.253:39243 requested a full tablet report, sending...
I20260806 22:50:15.275811 4741 heartbeater.cc:511] Master 127.28.218.254:44755 requested a full tablet report, sending...
I20260806 22:50:15.276185 3557 ts_manager.cc:194] Re-registered known tserver with Master: 05b84cde108549378d6584ef921fed23 (127.28.218.194:45405)
I20260806 22:50:15.276310 4190 ts_manager.cc:194] Registered new tserver with Master: 05b84cde108549378d6584ef921fed23 (127.28.218.194:45405)
I20260806 22:50:15.276718 3557 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.28.218.194@KRBTEST.COM'} at 127.28.218.194:47699
I20260806 22:50:15.276876 4746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:15.261797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.252:39503 (local address 127.28.218.194:38377)
0806 22:50:15.262442 (+ 645us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:15.262452 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:15.263605 (+ 1153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:15.263968 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:15.263972 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:15.264535 (+ 563us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:15.264854 (+ 319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:15.264860 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:15.265937 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:15.265938 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:15.267404 (+ 1466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:15.267407 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:15.268001 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:15.268002 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:15.268103 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:15.268899 (+ 796us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:50:15.268904 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:15.271934 (+ 3030us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:50:15.275562 (+ 3628us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:15.275563 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:15.275563 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:15.275962 (+ 399us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:15.276190 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:15.276191 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:15.276191 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:15.276250 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:15.276570 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:50:15.276571 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:50:15.276627 (+ 56us) client_negotiation.cc:769] Sending connection context
0806 22:50:15.276681 (+ 54us) client_negotiation.cc:240] Negotiation successful
0806 22:50:15.276770 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":607,"mutex_wait_us":137,"thread_start_us":271,"threads_started":1}
I20260806 22:50:15.276938 4601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:15.261826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.194:38377 (local address 127.28.218.252:39503)
0806 22:50:15.261983 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:15.261985 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:15.262527 (+ 542us) server_negotiation.cc:407] Connection header received
0806 22:50:15.263730 (+ 1203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:15.263731 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:15.263747 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:15.263778 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:15.264977 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:15.265777 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:15.267506 (+ 1729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:15.267870 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:15.267912 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:15.272043 (+ 4131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:50:15.272044 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:50:15.272044 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:50:15.272053 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:50:15.273773 (+ 1720us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:15.276056 (+ 2283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:15.276058 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:15.276058 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:15.276121 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:15.276352 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:15.276353 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:15.276353 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:15.276502 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:50:15.276568 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:15.276721 (+ 153us) server_negotiation.cc:299] Negotiation successful
0806 22:50:15.276823 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111}
I20260806 22:50:15.277698 4740 heartbeater.cc:347] Connected to a master server at 127.28.218.252:39503
I20260806 22:50:15.277758 4740 heartbeater.cc:464] Registering TS with master...
I20260806 22:50:15.277832 4740 heartbeater.cc:511] Master 127.28.218.252:39503 requested a full tablet report, sending...
I20260806 22:50:15.278187 4399 ts_manager.cc:194] Registered new tserver with Master: 05b84cde108549378d6584ef921fed23 (127.28.218.194:45405)
W20260806 22:50:15.361462 4747 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:50:15.361653 4747 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:50:15.361675 4747 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:50:15.362697 4747 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:50:15.362730 4747 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:50:15.362756 4747 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.218.195
I20260806 22:50:15.364230 4747 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:37527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/krb5kdc/oryx_127.28.218.195.keytab
--principal=oryx/127.28.218.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.218.195:42091
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.218.195
--webserver_port=46163
--tserver_master_addrs=127.28.218.254:44755,127.28.218.253:39243,127.28.218.252:39503
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4747
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.218.195
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:50:15.364408 4747 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:50:15.364588 4747 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:50:15.369474 4755 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:15.369534 4753 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:50:15.369724 4747 server_base.cc:1034] running on GCE node
W20260806 22:50:15.369772 4752 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:50:15.370018 4747 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:50:15.370265 4747 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:50:15.371398 4747 hybrid_clock.cc:648] HybridClock initialized: now 1786056615371379 us; error 34 us; skew 500 ppm
Aug 06 22:50:15 dist-test-slave-1xrh krb5kdc[3512](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056615, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.218.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:50:15.373849 4747 init.cc:377] Logged in from keytab as oryx/127.28.218.195@KRBTEST.COM (short username oryx)
I20260806 22:50:15.374388 4747 webserver.cc:460] Webserver started at http://127.28.218.195:46163/ using document root <none> and password file <none>
I20260806 22:50:15.374593 4747 fs_manager.cc:359] Metadata directory not provided
I20260806 22:50:15.374665 4747 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:50:15.376206 4747 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:50:15.376783 4762 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:15.376931 4747 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:50:15.377024 4747 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/ts-2/wal
uuid: "b11173b0207c46549ca94b7a987905cf"
format_stamp: "Formatted at 2026-08-06 22:50:14 on dist-test-slave-1xrh"
server_key: "793cf81257c3498a570ee9636982688f"
server_key_iv: "bd7cd6aaef60bce3ab4194010109b3d9"
server_key_version: "encryptionkey@0"
I20260806 22:50:15.377321 4747 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:50:15.390568 4747 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:50:15.392313 4747 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:50:15.392485 4747 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:50:15.392697 4747 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:50:15.392982 4747 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:50:15.393013 4747 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:15.393034 4747 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:50:15.393075 4747 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:15.398766 4747 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.195:42091
I20260806 22:50:15.398835 4884 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.195:42091 every 8 connection(s)
I20260806 22:50:15.399210 4747 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/ts-2/data/info.pb
I20260806 22:50:15.400935 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 4747
Aug 06 22:50:15 dist-test-slave-1xrh krb5kdc[3512](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056615, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.218.195@KRBTEST.COM for oryx/127.28.218.252@KRBTEST.COM
Aug 06 22:50:15 dist-test-slave-1xrh krb5kdc[3512](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056615, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.218.195@KRBTEST.COM for oryx/127.28.218.254@KRBTEST.COM
Aug 06 22:50:15 dist-test-slave-1xrh krb5kdc[3512](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056615, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.218.195@KRBTEST.COM for oryx/127.28.218.253@KRBTEST.COM
I20260806 22:50:15.414152 4892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:15.399908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.252:39503 (local address 127.28.218.195:35699)
0806 22:50:15.400083 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:15.400126 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:15.401705 (+ 1579us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:15.402035 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:15.402036 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:15.402665 (+ 629us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:15.402892 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:15.402897 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:15.404044 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:15.404045 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:15.405465 (+ 1420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:15.405467 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:15.405997 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:15.405999 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:15.406141 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:15.406823 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:50:15.406834 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:15.411071 (+ 4237us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:50:15.412986 (+ 1915us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:15.412987 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:15.412988 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:15.413227 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:15.413440 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:15.413441 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:15.413441 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:15.413495 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:15.413786 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:50:15.413786 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:50:15.413841 (+ 55us) client_negotiation.cc:769] Sending connection context
0806 22:50:15.413911 (+ 70us) client_negotiation.cc:240] Negotiation successful
0806 22:50:15.414016 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"mutex_wait_us":110,"thread_start_us":100,"threads_started":1}
I20260806 22:50:15.414315 4893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:15.400204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.253:39243 (local address 127.28.218.195:57445)
0806 22:50:15.400350 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:15.400359 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:15.401699 (+ 1340us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:15.402029 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:15.402032 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:15.402580 (+ 548us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:15.402894 (+ 314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:15.402900 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:15.404057 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:15.404058 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:15.405464 (+ 1406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:15.405467 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:15.406010 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:15.406012 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:15.406141 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:15.407236 (+ 1095us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:50:15.407241 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:15.410257 (+ 3016us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:50:15.412287 (+ 2030us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:15.412295 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:15.412297 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:15.412621 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:15.412850 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:15.412850 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:15.412851 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:15.413008 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:15.413314 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:50:15.413321 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:50:15.413488 (+ 167us) client_negotiation.cc:769] Sending connection context
0806 22:50:15.413605 (+ 117us) client_negotiation.cc:240] Negotiation successful
0806 22:50:15.413801 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"mutex_wait_us":105,"thread_start_us":75,"threads_started":1}
I20260806 22:50:15.414644 4230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:15.400296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.195:57445 (local address 127.28.218.253:39243)
0806 22:50:15.400342 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:15.400344 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:15.400405 (+ 61us) server_negotiation.cc:407] Connection header received
0806 22:50:15.401904 (+ 1499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:15.401906 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:15.401920 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:15.401972 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:15.403015 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:15.403935 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:15.405569 (+ 1634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:15.405891 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:15.405952 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:15.410366 (+ 4414us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:50:15.410367 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:50:15.410367 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:50:15.410375 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:50:15.412202 (+ 1827us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:15.412710 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:15.412712 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:15.412712 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:15.412775 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:15.413089 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:15.413089 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:15.413090 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:15.413223 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:50:15.413285 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:15.414455 (+ 1170us) server_negotiation.cc:299] Negotiation successful
0806 22:50:15.414536 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 22:50:15.414652 4601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:15.399979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.195:35699 (local address 127.28.218.252:39503)
0806 22:50:15.400021 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:15.400023 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:15.400279 (+ 256us) server_negotiation.cc:407] Connection header received
0806 22:50:15.401872 (+ 1593us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:15.401873 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:15.401886 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:15.401954 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:15.403017 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:15.403899 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:15.405558 (+ 1659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:15.405868 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:15.405934 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:15.411191 (+ 5257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:50:15.411193 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:50:15.411194 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:50:15.411201 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:50:15.412892 (+ 1691us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:15.413309 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:15.413310 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:15.413310 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:15.413369 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:15.413565 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:15.413566 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:15.413566 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:15.413698 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:50:15.413760 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:15.414459 (+ 699us) server_negotiation.cc:299] Negotiation successful
0806 22:50:15.414560 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 22:50:15.415053 4885 heartbeater.cc:347] Connected to a master server at 127.28.218.252:39503
I20260806 22:50:15.415166 4888 heartbeater.cc:347] Connected to a master server at 127.28.218.253:39243
I20260806 22:50:15.415176 4885 heartbeater.cc:464] Registering TS with master...
I20260806 22:50:15.415213 4888 heartbeater.cc:464] Registering TS with master...
I20260806 22:50:15.415344 3620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:15.399906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.195:38413 (local address 127.28.218.254:44755)
0806 22:50:15.399953 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:15.399955 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:15.400223 (+ 268us) server_negotiation.cc:407] Connection header received
0806 22:50:15.401830 (+ 1607us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:15.401832 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:15.401847 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:15.401881 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:15.403012 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:15.403899 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:15.405571 (+ 1672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:15.405893 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:15.405952 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:15.410370 (+ 4418us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:50:15.410371 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:50:15.410371 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:50:15.410379 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:50:15.413260 (+ 2881us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:15.413642 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:15.413643 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:15.413643 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:15.413694 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:15.413891 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:15.413892 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:15.413892 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:15.414019 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:50:15.414082 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:15.415161 (+ 1079us) server_negotiation.cc:299] Negotiation successful
0806 22:50:15.415243 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 22:50:15.415396 4885 heartbeater.cc:511] Master 127.28.218.252:39503 requested a full tablet report, sending...
I20260806 22:50:15.415396 4888 heartbeater.cc:511] Master 127.28.218.253:39243 requested a full tablet report, sending...
I20260806 22:50:15.415863 4891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:15.399904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:44755 (local address 127.28.218.195:38413)
0806 22:50:15.400154 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:15.400165 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:15.401691 (+ 1526us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:15.402028 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:15.402032 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:15.402584 (+ 552us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:15.402892 (+ 308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:15.402897 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:15.404044 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:15.404045 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:15.405464 (+ 1419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:15.405467 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:15.406012 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:15.406013 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:15.406141 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:15.406921 (+ 780us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:50:15.406925 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:15.410257 (+ 3332us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:50:15.413354 (+ 3097us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:15.413354 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:15.413355 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:15.413566 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:15.413765 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:15.413766 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:15.413766 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:15.413820 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:15.414108 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:50:15.414109 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:50:15.414147 (+ 38us) client_negotiation.cc:769] Sending connection context
0806 22:50:15.415655 (+ 1508us) client_negotiation.cc:240] Negotiation successful
0806 22:50:15.415758 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":223,"mutex_wait_us":169,"thread_start_us":92,"threads_started":1}
I20260806 22:50:15.415932 4399 ts_manager.cc:194] Registered new tserver with Master: b11173b0207c46549ca94b7a987905cf (127.28.218.195:42091)
I20260806 22:50:15.415971 4190 ts_manager.cc:194] Registered new tserver with Master: b11173b0207c46549ca94b7a987905cf (127.28.218.195:42091)
I20260806 22:50:15.416421 4886 heartbeater.cc:347] Connected to a master server at 127.28.218.254:44755
I20260806 22:50:15.416472 4886 heartbeater.cc:464] Registering TS with master...
I20260806 22:50:15.416546 4886 heartbeater.cc:511] Master 127.28.218.254:44755 requested a full tablet report, sending...
I20260806 22:50:15.416755 3557 ts_manager.cc:194] Re-registered known tserver with Master: b11173b0207c46549ca94b7a987905cf (127.28.218.195:42091)
I20260806 22:50:15.416909 4898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:15.402144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51872 (local address 127.28.218.254:44755)
0806 22:50:15.404320 (+ 2176us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:15.404322 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:15.404330 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:50:15.407696 (+ 3366us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:15.407697 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:15.407713 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:15.408014 (+ 301us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:15.408716 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:15.409523 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:15.410701 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:15.411063 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:15.411107 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:15.413307 (+ 2200us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:50:15.413308 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:50:15.413308 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:50:15.413315 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:50:15.415075 (+ 1760us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:15.415941 (+ 866us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:15.415942 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:15.415942 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:15.416001 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:15.416197 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:15.416198 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:15.416199 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:15.416330 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:50:15.416414 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:15.416721 (+ 307us) server_negotiation.cc:299] Negotiation successful
0806 22:50:15.416805 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2141,"thread_start_us":86,"threads_started":1}
I20260806 22:50:15.417021 4900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:15.402377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46428 (local address 127.28.218.253:39243)
0806 22:50:15.402509 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:15.402511 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:15.403318 (+ 807us) server_negotiation.cc:407] Connection header received
0806 22:50:15.403340 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:15.403341 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:15.403352 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:15.403544 (+ 192us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:15.404391 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:15.405137 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:15.406190 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:15.406469 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:15.406547 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:15.408088 (+ 1541us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:50:15.408089 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:50:15.408090 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:50:15.408097 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:50:15.409975 (+ 1878us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:15.410366 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:15.410367 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:15.410367 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:15.410481 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:15.416208 (+ 5727us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:15.416209 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:15.416209 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:15.416328 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:50:15.416427 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:15.416854 (+ 427us) server_negotiation.cc:299] Negotiation successful
0806 22:50:15.416934 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":76,"threads_started":1}
I20260806 22:50:15.417218 3557 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.28.218.195@KRBTEST.COM'} at 127.28.218.195:38413
I20260806 22:50:15.419759 3557 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51872:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 22:50:15.421351 4902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:15.402758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45982 (local address 127.28.218.252:39503)
0806 22:50:15.403012 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:15.403014 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:15.408147 (+ 5133us) server_negotiation.cc:407] Connection header received
0806 22:50:15.408258 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:15.408259 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:15.408271 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:15.408308 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:15.409257 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:15.410848 (+ 1591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:15.415682 (+ 4834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:15.416848 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:15.416893 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:15.418816 (+ 1923us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:50:15.418817 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:50:15.418818 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:50:15.418825 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:50:15.420180 (+ 1355us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:15.420578 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:15.420580 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:15.420580 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:15.420647 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:15.420841 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:15.420842 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:15.420843 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:15.420958 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:50:15.421055 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:15.421159 (+ 104us) server_negotiation.cc:299] Negotiation successful
0806 22:50:15.421239 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":233,"thread_start_us":60,"threads_started":1}
W20260806 22:50:15.421562 3557 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:50:15.429276 4908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:15.425290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.194:45405 (local address 127.0.0.1:45654)
0806 22:50:15.425533 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:15.425543 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:15.425603 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:15.426038 (+ 435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:15.426039 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:15.426041 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:50:15.426094 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:15.426096 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:15.427100 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:15.427101 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:15.428112 (+ 1011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:15.428114 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:15.429055 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:15.429055 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:15.429115 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:15.429119 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:50:15.429165 (+ 46us) client_negotiation.cc:240] Negotiation successful
0806 22:50:15.429204 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":225,"mutex_wait_us":114,"thread_start_us":86,"threads_started":1}
I20260806 22:50:15.429311 4912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:15.425702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45654 (local address 127.28.218.194:45405)
0806 22:50:15.425880 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:15.425884 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:15.425893 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:50:15.425931 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:15.425933 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:15.425953 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:15.426019 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:50:15.426173 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:15.427018 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:15.428207 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:15.428979 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:15.429072 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:15.429127 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:15.429189 (+ 62us) server_negotiation.cc:299] Negotiation successful
0806 22:50:15.429228 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":114,"threads_started":1}
I20260806 22:50:15.430042 4910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:15.425597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.193:35729 (local address 127.0.0.1:60830)
0806 22:50:15.425720 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:15.425729 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:15.425770 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:15.426399 (+ 629us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:15.426400 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:15.426402 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:50:15.426444 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:15.426445 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:15.428130 (+ 1685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:15.428131 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:15.428891 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:15.428893 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:15.429804 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:15.429805 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:15.429865 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:15.429868 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:50:15.429931 (+ 63us) client_negotiation.cc:240] Negotiation successful
0806 22:50:15.429971 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":82,"threads_started":1}
I20260806 22:50:15.430066 4911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:15.425696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60830 (local address 127.28.218.193:35729)
0806 22:50:15.426246 (+ 550us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:15.426250 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:15.426259 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:50:15.426296 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:15.426298 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:15.426313 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:15.426391 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:50:15.426507 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:15.428050 (+ 1543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:15.428982 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:15.429716 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:15.429823 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:15.429878 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:15.429950 (+ 72us) server_negotiation.cc:299] Negotiation successful
0806 22:50:15.429990 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":508,"thread_start_us":75,"threads_started":1}
I20260806 22:50:15.430151 4148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:15.425289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.195:42091 (local address 127.0.0.1:43164)
0806 22:50:15.425326 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:15.425338 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:15.425397 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:15.426112 (+ 715us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:15.426113 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:15.426115 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:50:15.426169 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:15.426170 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:15.427671 (+ 1501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:15.427672 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:15.428705 (+ 1033us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:15.428708 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:15.429947 (+ 1239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:15.429949 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:15.430007 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:15.430011 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:50:15.430040 (+ 29us) client_negotiation.cc:240] Negotiation successful
0806 22:50:15.430083 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21}
I20260806 22:50:15.430138 4668 tablet_service.cc:1467] Processing CreateTablet for tablet dad92886214d4333bdb8454d3b546d29 (DEFAULT_TABLE table=test-table [id=a674babd68154fb2bb4984d2d1c2200a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:50:15.430537 4668 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dad92886214d4333bdb8454d3b546d29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:50:15.430598 4522 tablet_service.cc:1467] Processing CreateTablet for tablet dad92886214d4333bdb8454d3b546d29 (DEFAULT_TABLE table=test-table [id=a674babd68154fb2bb4984d2d1c2200a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:50:15.430840 4522 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dad92886214d4333bdb8454d3b546d29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:50:15.431044 4909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:15.425438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43164 (local address 127.28.218.195:42091)
0806 22:50:15.425947 (+ 509us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:15.425952 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:15.425962 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:50:15.426014 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:15.426016 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:15.426032 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:15.426124 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:50:15.426240 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:15.427596 (+ 1356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:15.428794 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:15.429839 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:15.430777 (+ 938us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:15.430835 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:15.430922 (+ 87us) server_negotiation.cc:299] Negotiation successful
0806 22:50:15.430964 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":477,"thread_start_us":166,"threads_started":1}
I20260806 22:50:15.431916 4814 tablet_service.cc:1467] Processing CreateTablet for tablet dad92886214d4333bdb8454d3b546d29 (DEFAULT_TABLE table=test-table [id=a674babd68154fb2bb4984d2d1c2200a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:50:15.432376 4814 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dad92886214d4333bdb8454d3b546d29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:50:15.433480 4913 tablet_bootstrap.cc:492] T dad92886214d4333bdb8454d3b546d29 P 7785c46d26114840970c8f163edf0196: Bootstrap starting.
I20260806 22:50:15.434509 4913 tablet_bootstrap.cc:654] T dad92886214d4333bdb8454d3b546d29 P 7785c46d26114840970c8f163edf0196: Neither blocks nor log segments found. Creating new log.
I20260806 22:50:15.434700 4914 tablet_bootstrap.cc:492] T dad92886214d4333bdb8454d3b546d29 P 05b84cde108549378d6584ef921fed23: Bootstrap starting.
I20260806 22:50:15.434831 4913 log.cc:824] T dad92886214d4333bdb8454d3b546d29 P 7785c46d26114840970c8f163edf0196: Log is configured to *not* fsync() on all Append() calls
I20260806 22:50:15.435353 4914 tablet_bootstrap.cc:654] T dad92886214d4333bdb8454d3b546d29 P 05b84cde108549378d6584ef921fed23: Neither blocks nor log segments found. Creating new log.
I20260806 22:50:15.435681 4914 log.cc:824] T dad92886214d4333bdb8454d3b546d29 P 05b84cde108549378d6584ef921fed23: Log is configured to *not* fsync() on all Append() calls
I20260806 22:50:15.435799 4913 tablet_bootstrap.cc:492] T dad92886214d4333bdb8454d3b546d29 P 7785c46d26114840970c8f163edf0196: No bootstrap required, opened a new log
I20260806 22:50:15.435876 4913 ts_tablet_manager.cc:1397] T dad92886214d4333bdb8454d3b546d29 P 7785c46d26114840970c8f163edf0196: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:50:15.436020 4918 tablet_bootstrap.cc:492] T dad92886214d4333bdb8454d3b546d29 P b11173b0207c46549ca94b7a987905cf: Bootstrap starting.
I20260806 22:50:15.436636 4918 tablet_bootstrap.cc:654] T dad92886214d4333bdb8454d3b546d29 P b11173b0207c46549ca94b7a987905cf: Neither blocks nor log segments found. Creating new log.
I20260806 22:50:15.437085 4918 log.cc:824] T dad92886214d4333bdb8454d3b546d29 P b11173b0207c46549ca94b7a987905cf: Log is configured to *not* fsync() on all Append() calls
I20260806 22:50:15.437865 4918 tablet_bootstrap.cc:492] T dad92886214d4333bdb8454d3b546d29 P b11173b0207c46549ca94b7a987905cf: No bootstrap required, opened a new log
I20260806 22:50:15.437947 4918 ts_tablet_manager.cc:1397] T dad92886214d4333bdb8454d3b546d29 P b11173b0207c46549ca94b7a987905cf: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:50:15.438236 4913 raft_consensus.cc:348] T dad92886214d4333bdb8454d3b546d29 P 7785c46d26114840970c8f163edf0196 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b84cde108549378d6584ef921fed23" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 45405 } } peers { permanent_uuid: "b11173b0207c46549ca94b7a987905cf" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 42091 } } peers { permanent_uuid: "7785c46d26114840970c8f163edf0196" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 35729 } }
I20260806 22:50:15.438403 4913 raft_consensus.cc:374] T dad92886214d4333bdb8454d3b546d29 P 7785c46d26114840970c8f163edf0196 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:50:15.438445 4913 raft_consensus.cc:729] T dad92886214d4333bdb8454d3b546d29 P 7785c46d26114840970c8f163edf0196 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7785c46d26114840970c8f163edf0196, State: Initialized, Role: FOLLOWER
I20260806 22:50:15.438568 4913 consensus_queue.cc:260] T dad92886214d4333bdb8454d3b546d29 P 7785c46d26114840970c8f163edf0196 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b84cde108549378d6584ef921fed23" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 45405 } } peers { permanent_uuid: "b11173b0207c46549ca94b7a987905cf" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 42091 } } peers { permanent_uuid: "7785c46d26114840970c8f163edf0196" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 35729 } }
I20260806 22:50:15.438827 4913 ts_tablet_manager.cc:1428] T dad92886214d4333bdb8454d3b546d29 P 7785c46d26114840970c8f163edf0196: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 22:50:15.438891 4596 heartbeater.cc:503] Master 127.28.218.254:44755 was elected leader, sending a full tablet report...
I20260806 22:50:15.439764 4914 tablet_bootstrap.cc:492] T dad92886214d4333bdb8454d3b546d29 P 05b84cde108549378d6584ef921fed23: No bootstrap required, opened a new log
I20260806 22:50:15.439833 4914 ts_tablet_manager.cc:1397] T dad92886214d4333bdb8454d3b546d29 P 05b84cde108549378d6584ef921fed23: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260806 22:50:15.440689 4918 raft_consensus.cc:348] T dad92886214d4333bdb8454d3b546d29 P b11173b0207c46549ca94b7a987905cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b84cde108549378d6584ef921fed23" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 45405 } } peers { permanent_uuid: "b11173b0207c46549ca94b7a987905cf" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 42091 } } peers { permanent_uuid: "7785c46d26114840970c8f163edf0196" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 35729 } }
I20260806 22:50:15.440869 4918 raft_consensus.cc:374] T dad92886214d4333bdb8454d3b546d29 P b11173b0207c46549ca94b7a987905cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:50:15.440936 4918 raft_consensus.cc:729] T dad92886214d4333bdb8454d3b546d29 P b11173b0207c46549ca94b7a987905cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b11173b0207c46549ca94b7a987905cf, State: Initialized, Role: FOLLOWER
I20260806 22:50:15.441052 4918 consensus_queue.cc:260] T dad92886214d4333bdb8454d3b546d29 P b11173b0207c46549ca94b7a987905cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b84cde108549378d6584ef921fed23" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 45405 } } peers { permanent_uuid: "b11173b0207c46549ca94b7a987905cf" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 42091 } } peers { permanent_uuid: "7785c46d26114840970c8f163edf0196" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 35729 } }
I20260806 22:50:15.441335 4918 ts_tablet_manager.cc:1428] T dad92886214d4333bdb8454d3b546d29 P b11173b0207c46549ca94b7a987905cf: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260806 22:50:15.441385 4886 heartbeater.cc:503] Master 127.28.218.254:44755 was elected leader, sending a full tablet report...
I20260806 22:50:15.442663 4914 raft_consensus.cc:348] T dad92886214d4333bdb8454d3b546d29 P 05b84cde108549378d6584ef921fed23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b84cde108549378d6584ef921fed23" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 45405 } } peers { permanent_uuid: "b11173b0207c46549ca94b7a987905cf" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 42091 } } peers { permanent_uuid: "7785c46d26114840970c8f163edf0196" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 35729 } }
I20260806 22:50:15.442808 4914 raft_consensus.cc:374] T dad92886214d4333bdb8454d3b546d29 P 05b84cde108549378d6584ef921fed23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:50:15.442854 4914 raft_consensus.cc:729] T dad92886214d4333bdb8454d3b546d29 P 05b84cde108549378d6584ef921fed23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05b84cde108549378d6584ef921fed23, State: Initialized, Role: FOLLOWER
I20260806 22:50:15.442970 4914 consensus_queue.cc:260] T dad92886214d4333bdb8454d3b546d29 P 05b84cde108549378d6584ef921fed23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b84cde108549378d6584ef921fed23" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 45405 } } peers { permanent_uuid: "b11173b0207c46549ca94b7a987905cf" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 42091 } } peers { permanent_uuid: "7785c46d26114840970c8f163edf0196" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 35729 } }
I20260806 22:50:15.443243 4914 ts_tablet_manager.cc:1428] T dad92886214d4333bdb8454d3b546d29 P 05b84cde108549378d6584ef921fed23: Time spent starting tablet: real 0.003s user 0.002s sys 0.002s
I20260806 22:50:15.443288 4741 heartbeater.cc:503] Master 127.28.218.254:44755 was elected leader, sending a full tablet report...
I20260806 22:50:15.480700 4902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:15.477840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45984 (local address 127.28.218.252:39503)
0806 22:50:15.477901 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:15.477903 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:15.477912 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:50:15.477975 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:15.477976 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:15.478003 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:15.478078 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:50:15.478211 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:15.478804 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:15.479995 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:15.480454 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:15.480513 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:15.480568 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:15.480616 (+ 48us) server_negotiation.cc:299] Negotiation successful
0806 22:50:15.480643 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 22:50:15.480701 4148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:15.477770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.252:39503 (local address 127.0.0.1:45984)
0806 22:50:15.477832 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:15.477847 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:15.477902 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:15.478092 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:15.478093 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:15.478095 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:50:15.478153 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:15.478154 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:15.478917 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:15.478917 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:15.479913 (+ 996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:15.479916 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:15.480514 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:15.480515 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:15.480559 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:15.480563 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:50:15.480610 (+ 47us) client_negotiation.cc:240] Negotiation successful
0806 22:50:15.480643 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260806 22:50:15.481055 4412 raft_consensus.cc:1203] T 00000000000000000000000000000000 P ed70ecfd24614dbc990e056d21844d64 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.9] Dedup: 1.7->[1.8-1.9]
I20260806 22:50:15.481477 4446 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed70ecfd24614dbc990e056d21844d64 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 20186b499bd34a7d9d497ba785db0fc9. Latest consensus state: current_term: 1 leader_uuid: "20186b499bd34a7d9d497ba785db0fc9" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "20186b499bd34a7d9d497ba785db0fc9" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 44755 } } peers { permanent_uuid: "eb135056735a446dbff018e4072cab0c" member_type: VOTER last_known_addr { host: "127.28.218.253" port: 39243 } attrs { promote: false } } peers { permanent_uuid: "ed70ecfd24614dbc990e056d21844d64" member_type: NON_VOTER last_known_addr { host: "127.28.218.252" port: 39503 } attrs { promote: true } } }
I20260806 22:50:15.481585 4446 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed70ecfd24614dbc990e056d21844d64 [sys.catalog]: This master's current role is: LEARNER
W20260806 22:50:15.512324 4743 tablet.cc:2369] T dad92886214d4333bdb8454d3b546d29 P 05b84cde108549378d6584ef921fed23: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:50:15.517817 4921 raft_consensus.cc:482] T dad92886214d4333bdb8454d3b546d29 P b11173b0207c46549ca94b7a987905cf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:50:15.517976 4921 raft_consensus.cc:504] T dad92886214d4333bdb8454d3b546d29 P b11173b0207c46549ca94b7a987905cf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b84cde108549378d6584ef921fed23" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 45405 } } peers { permanent_uuid: "b11173b0207c46549ca94b7a987905cf" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 42091 } } peers { permanent_uuid: "7785c46d26114840970c8f163edf0196" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 35729 } }
I20260806 22:50:15.518366 4921 leader_election.cc:288] T dad92886214d4333bdb8454d3b546d29 P b11173b0207c46549ca94b7a987905cf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 05b84cde108549378d6584ef921fed23 (127.28.218.194:45405), 7785c46d26114840970c8f163edf0196 (127.28.218.193:35729)
I20260806 22:50:15.521242 4891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:15.518541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.194:45405 (local address 127.28.218.195:45953)
0806 22:50:15.518599 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:15.518611 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:15.518662 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:15.518808 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:15.518810 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:15.518812 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:50:15.518890 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:15.518892 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:15.519639 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:15.519640 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:15.520430 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:15.520432 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:15.520996 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:15.520997 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:15.521074 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:15.521078 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:50:15.521131 (+ 53us) client_negotiation.cc:240] Negotiation successful
0806 22:50:15.521189 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260806 22:50:15.521332 4912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:15.518629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.195:45953 (local address 127.28.218.194:45405)
0806 22:50:15.518684 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:15.518686 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:15.518707 (+ 21us) server_negotiation.cc:407] Connection header received
0806 22:50:15.518735 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:15.518736 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:15.518746 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:15.518795 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:50:15.518965 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:15.519566 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:15.520500 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:15.520927 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:15.520982 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:15.520989 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:15.521191 (+ 202us) server_negotiation.cc:299] Negotiation successful
0806 22:50:15.521230 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 22:50:15.521409 4892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:15.518678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.193:35729 (local address 127.28.218.195:42435)
0806 22:50:15.518722 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:15.518731 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:15.518757 (+ 26us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:15.518907 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:15.518907 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:15.518908 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:50:15.518997 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:15.518999 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:15.519692 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:15.519693 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:15.520526 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:15.520528 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:15.521142 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:15.521143 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:15.521238 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:15.521241 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:50:15.521300 (+ 59us) client_negotiation.cc:240] Negotiation successful
0806 22:50:15.521339 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260806 22:50:15.521481 4911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:15.518745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.195:42435 (local address 127.28.218.193:35729)
0806 22:50:15.518790 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:15.518792 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:15.518801 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:50:15.518826 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:15.518827 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:15.518835 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:15.518900 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:50:15.519066 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:15.519615 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:15.520622 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:15.521041 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:15.521113 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:15.521122 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:15.521318 (+ 196us) server_negotiation.cc:299] Negotiation successful
0806 22:50:15.521365 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 22:50:15.521852 4691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad92886214d4333bdb8454d3b546d29" candidate_uuid: "b11173b0207c46549ca94b7a987905cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05b84cde108549378d6584ef921fed23" is_pre_election: true
I20260806 22:50:15.522007 4691 raft_consensus.cc:2454] T dad92886214d4333bdb8454d3b546d29 P 05b84cde108549378d6584ef921fed23 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b11173b0207c46549ca94b7a987905cf in term 0.
I20260806 22:50:15.522024 4543 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad92886214d4333bdb8454d3b546d29" candidate_uuid: "b11173b0207c46549ca94b7a987905cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7785c46d26114840970c8f163edf0196" is_pre_election: true
I20260806 22:50:15.522186 4543 raft_consensus.cc:2454] T dad92886214d4333bdb8454d3b546d29 P 7785c46d26114840970c8f163edf0196 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b11173b0207c46549ca94b7a987905cf in term 0.
I20260806 22:50:15.522254 4763 leader_election.cc:302] T dad92886214d4333bdb8454d3b546d29 P b11173b0207c46549ca94b7a987905cf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05b84cde108549378d6584ef921fed23, b11173b0207c46549ca94b7a987905cf; no voters:
I20260806 22:50:15.522385 4921 raft_consensus.cc:2781] T dad92886214d4333bdb8454d3b546d29 P b11173b0207c46549ca94b7a987905cf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:50:15.522449 4921 raft_consensus.cc:482] T dad92886214d4333bdb8454d3b546d29 P b11173b0207c46549ca94b7a987905cf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:50:15.522475 4921 raft_consensus.cc:3037] T dad92886214d4333bdb8454d3b546d29 P b11173b0207c46549ca94b7a987905cf [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:50:15.523347 4921 raft_consensus.cc:504] T dad92886214d4333bdb8454d3b546d29 P b11173b0207c46549ca94b7a987905cf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b84cde108549378d6584ef921fed23" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 45405 } } peers { permanent_uuid: "b11173b0207c46549ca94b7a987905cf" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 42091 } } peers { permanent_uuid: "7785c46d26114840970c8f163edf0196" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 35729 } }
I20260806 22:50:15.523473 4921 leader_election.cc:288] T dad92886214d4333bdb8454d3b546d29 P b11173b0207c46549ca94b7a987905cf [CANDIDATE]: Term 1 election: Requested vote from peers 05b84cde108549378d6584ef921fed23 (127.28.218.194:45405), 7785c46d26114840970c8f163edf0196 (127.28.218.193:35729)
I20260806 22:50:15.523645 4691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad92886214d4333bdb8454d3b546d29" candidate_uuid: "b11173b0207c46549ca94b7a987905cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05b84cde108549378d6584ef921fed23"
I20260806 22:50:15.523645 4543 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad92886214d4333bdb8454d3b546d29" candidate_uuid: "b11173b0207c46549ca94b7a987905cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7785c46d26114840970c8f163edf0196"
I20260806 22:50:15.523720 4691 raft_consensus.cc:3037] T dad92886214d4333bdb8454d3b546d29 P 05b84cde108549378d6584ef921fed23 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:50:15.523721 4543 raft_consensus.cc:3037] T dad92886214d4333bdb8454d3b546d29 P 7785c46d26114840970c8f163edf0196 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:50:15.524736 4543 raft_consensus.cc:2454] T dad92886214d4333bdb8454d3b546d29 P 7785c46d26114840970c8f163edf0196 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b11173b0207c46549ca94b7a987905cf in term 1.
I20260806 22:50:15.524736 4691 raft_consensus.cc:2454] T dad92886214d4333bdb8454d3b546d29 P 05b84cde108549378d6584ef921fed23 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b11173b0207c46549ca94b7a987905cf in term 1.
I20260806 22:50:15.524902 4763 leader_election.cc:302] T dad92886214d4333bdb8454d3b546d29 P b11173b0207c46549ca94b7a987905cf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7785c46d26114840970c8f163edf0196, b11173b0207c46549ca94b7a987905cf; no voters:
I20260806 22:50:15.524994 4921 raft_consensus.cc:2781] T dad92886214d4333bdb8454d3b546d29 P b11173b0207c46549ca94b7a987905cf [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:50:15.525177 4921 raft_consensus.cc:686] T dad92886214d4333bdb8454d3b546d29 P b11173b0207c46549ca94b7a987905cf [term 1 LEADER]: Becoming Leader. State: Replica: b11173b0207c46549ca94b7a987905cf, State: Running, Role: LEADER
I20260806 22:50:15.525321 4921 consensus_queue.cc:237] T dad92886214d4333bdb8454d3b546d29 P b11173b0207c46549ca94b7a987905cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b84cde108549378d6584ef921fed23" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 45405 } } peers { permanent_uuid: "b11173b0207c46549ca94b7a987905cf" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 42091 } } peers { permanent_uuid: "7785c46d26114840970c8f163edf0196" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 35729 } }
I20260806 22:50:15.526077 3557 catalog_manager.cc:5495] T dad92886214d4333bdb8454d3b546d29 P b11173b0207c46549ca94b7a987905cf reported cstate change: term changed from 0 to 1, leader changed from <none> to b11173b0207c46549ca94b7a987905cf (127.28.218.195). New cstate: current_term: 1 leader_uuid: "b11173b0207c46549ca94b7a987905cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b84cde108549378d6584ef921fed23" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 45405 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b11173b0207c46549ca94b7a987905cf" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 42091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7785c46d26114840970c8f163edf0196" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 35729 } health_report { overall_health: UNKNOWN } } }
I20260806 22:50:15.526722 4151 sys_catalog.cc:455] T 00000000000000000000000000000000 P 20186b499bd34a7d9d497ba785db0fc9 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "20186b499bd34a7d9d497ba785db0fc9" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "20186b499bd34a7d9d497ba785db0fc9" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 44755 } } peers { permanent_uuid: "eb135056735a446dbff018e4072cab0c" member_type: VOTER last_known_addr { host: "127.28.218.253" port: 39243 } attrs { promote: false } } peers { permanent_uuid: "ed70ecfd24614dbc990e056d21844d64" member_type: NON_VOTER last_known_addr { host: "127.28.218.252" port: 39503 } attrs { promote: true } } }
I20260806 22:50:15.526813 4151 sys_catalog.cc:458] T 00000000000000000000000000000000 P 20186b499bd34a7d9d497ba785db0fc9 [sys.catalog]: This master's current role is: LEADER
I20260806 22:50:15.527686 4150 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 20186b499bd34a7d9d497ba785db0fc9: attempting to promote NON_VOTER ed70ecfd24614dbc990e056d21844d64 to VOTER
I20260806 22:50:15.527810 4150 consensus_queue.cc:237] T 00000000000000000000000000000000 P 20186b499bd34a7d9d497ba785db0fc9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 1.10, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "20186b499bd34a7d9d497ba785db0fc9" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 44755 } } peers { permanent_uuid: "eb135056735a446dbff018e4072cab0c" member_type: VOTER last_known_addr { host: "127.28.218.253" port: 39243 } attrs { promote: false } } peers { permanent_uuid: "ed70ecfd24614dbc990e056d21844d64" member_type: VOTER last_known_addr { host: "127.28.218.252" port: 39503 } attrs { promote: false } }
I20260806 22:50:15.528260 4412 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ed70ecfd24614dbc990e056d21844d64 [term 1 LEARNER]: Refusing update from remote peer 20186b499bd34a7d9d497ba785db0fc9: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260806 22:50:15.528297 4200 raft_consensus.cc:1261] T 00000000000000000000000000000000 P eb135056735a446dbff018e4072cab0c [term 1 FOLLOWER]: Refusing update from remote peer 20186b499bd34a7d9d497ba785db0fc9: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260806 22:50:15.528462 4151 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 20186b499bd34a7d9d497ba785db0fc9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed70ecfd24614dbc990e056d21844d64" member_type: VOTER last_known_addr { host: "127.28.218.252" port: 39503 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260806 22:50:15.528553 4149 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 20186b499bd34a7d9d497ba785db0fc9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb135056735a446dbff018e4072cab0c" member_type: VOTER last_known_addr { host: "127.28.218.253" port: 39243 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260806 22:50:15.529114 4149 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 20186b499bd34a7d9d497ba785db0fc9 [term 1 LEADER]: Committing config change with OpId 1.11: config changed from index 7 to 11, ed70ecfd24614dbc990e056d21844d64 (127.28.218.252) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "20186b499bd34a7d9d497ba785db0fc9" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 44755 } } peers { permanent_uuid: "eb135056735a446dbff018e4072cab0c" member_type: VOTER last_known_addr { host: "127.28.218.253" port: 39243 } attrs { promote: false } } peers { permanent_uuid: "ed70ecfd24614dbc990e056d21844d64" member_type: VOTER last_known_addr { host: "127.28.218.252" port: 39503 } attrs { promote: false } } }
I20260806 22:50:15.529289 4200 raft_consensus.cc:2932] T 00000000000000000000000000000000 P eb135056735a446dbff018e4072cab0c [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, ed70ecfd24614dbc990e056d21844d64 (127.28.218.252) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "20186b499bd34a7d9d497ba785db0fc9" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 44755 } } peers { permanent_uuid: "eb135056735a446dbff018e4072cab0c" member_type: VOTER last_known_addr { host: "127.28.218.253" port: 39243 } attrs { promote: false } } peers { permanent_uuid: "ed70ecfd24614dbc990e056d21844d64" member_type: VOTER last_known_addr { host: "127.28.218.252" port: 39503 } attrs { promote: false } } }
I20260806 22:50:15.529299 4412 raft_consensus.cc:2932] T 00000000000000000000000000000000 P ed70ecfd24614dbc990e056d21844d64 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, ed70ecfd24614dbc990e056d21844d64 (127.28.218.252) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "20186b499bd34a7d9d497ba785db0fc9" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 44755 } } peers { permanent_uuid: "eb135056735a446dbff018e4072cab0c" member_type: VOTER last_known_addr { host: "127.28.218.253" port: 39243 } attrs { promote: false } } peers { permanent_uuid: "ed70ecfd24614dbc990e056d21844d64" member_type: VOTER last_known_addr { host: "127.28.218.252" port: 39503 } attrs { promote: false } } }
I20260806 22:50:15.530063 4149 sys_catalog.cc:455] T 00000000000000000000000000000000 P 20186b499bd34a7d9d497ba785db0fc9 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "20186b499bd34a7d9d497ba785db0fc9" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "20186b499bd34a7d9d497ba785db0fc9" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 44755 } } peers { permanent_uuid: "eb135056735a446dbff018e4072cab0c" member_type: VOTER last_known_addr { host: "127.28.218.253" port: 39243 } attrs { promote: false } } peers { permanent_uuid: "ed70ecfd24614dbc990e056d21844d64" member_type: VOTER last_known_addr { host: "127.28.218.252" port: 39503 } attrs { promote: false } } }
I20260806 22:50:15.530140 4149 sys_catalog.cc:458] T 00000000000000000000000000000000 P 20186b499bd34a7d9d497ba785db0fc9 [sys.catalog]: This master's current role is: LEADER
I20260806 22:50:15.530187 4149 sys_catalog.cc:455] T 00000000000000000000000000000000 P 20186b499bd34a7d9d497ba785db0fc9 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "20186b499bd34a7d9d497ba785db0fc9" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "20186b499bd34a7d9d497ba785db0fc9" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 44755 } } peers { permanent_uuid: "eb135056735a446dbff018e4072cab0c" member_type: VOTER last_known_addr { host: "127.28.218.253" port: 39243 } attrs { promote: false } } peers { permanent_uuid: "ed70ecfd24614dbc990e056d21844d64" member_type: VOTER last_known_addr { host: "127.28.218.252" port: 39503 } attrs { promote: false } } }
I20260806 22:50:15.530251 4149 sys_catalog.cc:458] T 00000000000000000000000000000000 P 20186b499bd34a7d9d497ba785db0fc9 [sys.catalog]: This master's current role is: LEADER
I20260806 22:50:15.530210 4929 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb135056735a446dbff018e4072cab0c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "20186b499bd34a7d9d497ba785db0fc9" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "20186b499bd34a7d9d497ba785db0fc9" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 44755 } } peers { permanent_uuid: "eb135056735a446dbff018e4072cab0c" member_type: VOTER last_known_addr { host: "127.28.218.253" port: 39243 } attrs { promote: false } } peers { permanent_uuid: "ed70ecfd24614dbc990e056d21844d64" member_type: VOTER last_known_addr { host: "127.28.218.252" port: 39503 } attrs { promote: false } } }
I20260806 22:50:15.530145 4151 sys_catalog.cc:455] T 00000000000000000000000000000000 P 20186b499bd34a7d9d497ba785db0fc9 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "20186b499bd34a7d9d497ba785db0fc9" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "20186b499bd34a7d9d497ba785db0fc9" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 44755 } } peers { permanent_uuid: "eb135056735a446dbff018e4072cab0c" member_type: VOTER last_known_addr { host: "127.28.218.253" port: 39243 } attrs { promote: false } } peers { permanent_uuid: "ed70ecfd24614dbc990e056d21844d64" member_type: VOTER last_known_addr { host: "127.28.218.252" port: 39503 } attrs { promote: false } } }
I20260806 22:50:15.530292 4151 sys_catalog.cc:458] T 00000000000000000000000000000000 P 20186b499bd34a7d9d497ba785db0fc9 [sys.catalog]: This master's current role is: LEADER
I20260806 22:50:15.530292 4929 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb135056735a446dbff018e4072cab0c [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:50:15.530405 4446 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed70ecfd24614dbc990e056d21844d64 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "20186b499bd34a7d9d497ba785db0fc9" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "20186b499bd34a7d9d497ba785db0fc9" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 44755 } } peers { permanent_uuid: "eb135056735a446dbff018e4072cab0c" member_type: VOTER last_known_addr { host: "127.28.218.253" port: 39243 } attrs { promote: false } } peers { permanent_uuid: "ed70ecfd24614dbc990e056d21844d64" member_type: VOTER last_known_addr { host: "127.28.218.252" port: 39503 } attrs { promote: false } } }
I20260806 22:50:15.530488 4446 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed70ecfd24614dbc990e056d21844d64 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:50:15.544405 4909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:15.542265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43166 (local address 127.28.218.195:42091)
0806 22:50:15.542316 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:15.542318 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:15.542341 (+ 23us) server_negotiation.cc:407] Connection header received
0806 22:50:15.542379 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:15.542379 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:15.542394 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:15.542446 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:50:15.542566 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:15.543041 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:15.543757 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:15.543983 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:15.544038 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:15.544153 (+ 115us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:50:15.544244 (+ 91us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:50:15.544305 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:15.544334 (+ 29us) server_negotiation.cc:299] Negotiation successful
0806 22:50:15.544352 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260806 22:50:15.546339 4543 raft_consensus.cc:1261] T dad92886214d4333bdb8454d3b546d29 P 7785c46d26114840970c8f163edf0196 [term 1 FOLLOWER]: Refusing update from remote peer b11173b0207c46549ca94b7a987905cf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:50:15.546345 4691 raft_consensus.cc:1261] T dad92886214d4333bdb8454d3b546d29 P 05b84cde108549378d6584ef921fed23 [term 1 FOLLOWER]: Refusing update from remote peer b11173b0207c46549ca94b7a987905cf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:50:15.546691 4926 consensus_queue.cc:1035] T dad92886214d4333bdb8454d3b546d29 P b11173b0207c46549ca94b7a987905cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "05b84cde108549378d6584ef921fed23" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 45405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:50:15.546802 4921 consensus_queue.cc:1035] T dad92886214d4333bdb8454d3b546d29 P b11173b0207c46549ca94b7a987905cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "7785c46d26114840970c8f163edf0196" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 35729 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:50:15.549422 4932 mvcc.cc:205] Tried to move back new op lower bound from 7315687897274990592 to 7315687897192304640. Current Snapshot: MvccSnapshot[applied={T|T < 7315687897274990592}]
I20260806 22:50:15.549887 4934 mvcc.cc:205] Tried to move back new op lower bound from 7315687897274990592 to 7315687897192304640. Current Snapshot: MvccSnapshot[applied={T|T < 7315687897274990592}]
I20260806 22:50:15.552572 3556 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51872:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:50:15.552657 3556 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51872:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:50:15.554467 3556 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 20186b499bd34a7d9d497ba785db0fc9: Sending DeleteTablet for 3 replicas of tablet dad92886214d4333bdb8454d3b546d29
I20260806 22:50:15.554778 4814 tablet_service.cc:1514] Processing DeleteTablet for tablet dad92886214d4333bdb8454d3b546d29 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:50:15 UTC) from {username='oryx', principal='oryx/127.28.218.254@KRBTEST.COM'} at 127.0.0.1:43164
I20260806 22:50:15.554795 4522 tablet_service.cc:1514] Processing DeleteTablet for tablet dad92886214d4333bdb8454d3b546d29 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:50:15 UTC) from {username='oryx', principal='oryx/127.28.218.254@KRBTEST.COM'} at 127.0.0.1:60830
I20260806 22:50:15.555008 4943 tablet_replica.cc:330] T dad92886214d4333bdb8454d3b546d29 P b11173b0207c46549ca94b7a987905cf: stopping tablet replica
I20260806 22:50:15.555002 4668 tablet_service.cc:1514] Processing DeleteTablet for tablet dad92886214d4333bdb8454d3b546d29 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:50:15 UTC) from {username='oryx', principal='oryx/127.28.218.254@KRBTEST.COM'} at 127.0.0.1:45654
I20260806 22:50:15.555111 4943 raft_consensus.cc:2229] T dad92886214d4333bdb8454d3b546d29 P b11173b0207c46549ca94b7a987905cf [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:50:15.555248 4943 raft_consensus.cc:2258] T dad92886214d4333bdb8454d3b546d29 P b11173b0207c46549ca94b7a987905cf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:50:15.555693 4943 ts_tablet_manager.cc:1905] T dad92886214d4333bdb8454d3b546d29 P b11173b0207c46549ca94b7a987905cf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:50:15.556444 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 3525
I20260806 22:50:15.559162 4943 ts_tablet_manager.cc:1918] T dad92886214d4333bdb8454d3b546d29 P b11173b0207c46549ca94b7a987905cf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:50:15.559247 4943 log.cc:1196] T dad92886214d4333bdb8454d3b546d29 P b11173b0207c46549ca94b7a987905cf: Deleting WAL directory at /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/ts-2/wal/wals/dad92886214d4333bdb8454d3b546d29
I20260806 22:50:15.559567 4943 ts_tablet_manager.cc:1939] T dad92886214d4333bdb8454d3b546d29 P b11173b0207c46549ca94b7a987905cf: Deleting consensus metadata
I20260806 22:50:15.563762 4902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:15.555576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46000 (local address 127.28.218.252:39503)
0806 22:50:15.555668 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:15.555670 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:15.555838 (+ 168us) server_negotiation.cc:407] Connection header received
0806 22:50:15.555936 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:15.555937 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:15.555948 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:15.555976 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:15.556894 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:15.557480 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:15.558292 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:15.558590 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:15.558630 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:15.560649 (+ 2019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:50:15.560651 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:50:15.560651 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:50:15.560666 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:50:15.562464 (+ 1798us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:15.562856 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:15.562857 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:15.562857 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:15.562919 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:15.563163 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:15.563164 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:15.563164 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:15.563305 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:50:15.563404 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:15.563545 (+ 141us) server_negotiation.cc:299] Negotiation successful
0806 22:50:15.563629 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
W20260806 22:50:15.565271 4767 connection.cc:535] server connection from 127.0.0.1:43164 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 22:50:15.566066 4945 tablet_replica.cc:330] T dad92886214d4333bdb8454d3b546d29 P 05b84cde108549378d6584ef921fed23: stopping tablet replica
I20260806 22:50:15.566182 4945 raft_consensus.cc:2229] T dad92886214d4333bdb8454d3b546d29 P 05b84cde108549378d6584ef921fed23 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:50:15.566282 4945 raft_consensus.cc:2258] T dad92886214d4333bdb8454d3b546d29 P 05b84cde108549378d6584ef921fed23 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:50:15.566555 4944 tablet_replica.cc:330] T dad92886214d4333bdb8454d3b546d29 P 7785c46d26114840970c8f163edf0196: stopping tablet replica
I20260806 22:50:15.566664 4944 raft_consensus.cc:2229] T dad92886214d4333bdb8454d3b546d29 P 7785c46d26114840970c8f163edf0196 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:50:15.566763 4944 raft_consensus.cc:2258] T dad92886214d4333bdb8454d3b546d29 P 7785c46d26114840970c8f163edf0196 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:50:15.567399 4945 ts_tablet_manager.cc:1905] T dad92886214d4333bdb8454d3b546d29 P 05b84cde108549378d6584ef921fed23: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:50:15.567430 4944 ts_tablet_manager.cc:1905] T dad92886214d4333bdb8454d3b546d29 P 7785c46d26114840970c8f163edf0196: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:50:15.569083 4945 ts_tablet_manager.cc:1918] T dad92886214d4333bdb8454d3b546d29 P 05b84cde108549378d6584ef921fed23: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:50:15.569173 4945 log.cc:1196] T dad92886214d4333bdb8454d3b546d29 P 05b84cde108549378d6584ef921fed23: Deleting WAL directory at /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/ts-1/wal/wals/dad92886214d4333bdb8454d3b546d29
I20260806 22:50:15.569500 4945 ts_tablet_manager.cc:1939] T dad92886214d4333bdb8454d3b546d29 P 05b84cde108549378d6584ef921fed23: Deleting consensus metadata
W20260806 22:50:15.569993 4621 connection.cc:441] server connection from 127.0.0.1:45654 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45654 (request call id 1) could send its response
I20260806 22:50:15.570183 4944 ts_tablet_manager.cc:1918] T dad92886214d4333bdb8454d3b546d29 P 7785c46d26114840970c8f163edf0196: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:50:15.570322 4944 log.cc:1196] T dad92886214d4333bdb8454d3b546d29 P 7785c46d26114840970c8f163edf0196: Deleting WAL directory at /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/ts-0/wal/wals/dad92886214d4333bdb8454d3b546d29
I20260806 22:50:15.570637 4944 ts_tablet_manager.cc:1939] T dad92886214d4333bdb8454d3b546d29 P 7785c46d26114840970c8f163edf0196: Deleting consensus metadata
W20260806 22:50:15.571003 4473 connection.cc:441] server connection from 127.0.0.1:60830 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:60830 (request call id 1) could send its response
I20260806 22:50:15.575484 4900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:15.566989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46444 (local address 127.28.218.253:39243)
0806 22:50:15.567042 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:15.567044 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:15.567590 (+ 546us) server_negotiation.cc:407] Connection header received
0806 22:50:15.567681 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:15.567682 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:15.567694 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:15.567722 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:15.568761 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:15.569508 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:15.570611 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:15.570922 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:15.570962 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:15.572538 (+ 1576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:50:15.572540 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:50:15.572540 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:50:15.572548 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:50:15.574112 (+ 1564us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:15.574465 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:15.574466 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:15.574466 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:15.574532 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:15.574797 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:15.574799 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:15.574799 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:15.574941 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:50:15.575002 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:15.575264 (+ 262us) server_negotiation.cc:299] Negotiation successful
0806 22:50:15.575360 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260806 22:50:15.576730 4902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:15.567271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46002 (local address 127.28.218.252:39503)
0806 22:50:15.567305 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:15.567307 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:15.569098 (+ 1791us) server_negotiation.cc:407] Connection header received
0806 22:50:15.569207 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:15.569208 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:15.569219 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:15.569248 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:15.570021 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:15.570709 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:15.571749 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:15.571953 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:15.572034 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:15.573638 (+ 1604us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:50:15.573640 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:50:15.573640 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:50:15.573648 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:50:15.575309 (+ 1661us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:15.575881 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:15.575882 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:15.575883 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:15.575966 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:15.576239 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:15.576240 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:15.576241 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:15.576383 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:50:15.576492 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:15.576566 (+ 74us) server_negotiation.cc:299] Negotiation successful
0806 22:50:15.576640 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 22:50:16.550879 4954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:16.550561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:44755 (local address 127.28.218.193:57387)
0806 22:50:16.550706 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:16.550814 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:44755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":79,"threads_started":1}
W20260806 22:50:16.551280 4596 heartbeater.cc:644] Failed to heartbeat to 127.28.218.254:44755 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.28.218.254:44755: connect: Connection refused (error 111)
I20260806 22:50:16.551491 4955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:16.551022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:44755 (local address 127.28.218.195:58327)
0806 22:50:16.551305 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:16.551445 (+ 140us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:44755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":58,"threads_started":1}
W20260806 22:50:16.551714 4886 heartbeater.cc:644] Failed to heartbeat to 127.28.218.254:44755 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.28.218.254:44755: connect: Connection refused (error 111)
I20260806 22:50:16.552507 4956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:16.552223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:44755 (local address 127.28.218.194:58381)
0806 22:50:16.552368 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:16.552443 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:44755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":79,"threads_started":1}
W20260806 22:50:16.552712 4741 heartbeater.cc:644] Failed to heartbeat to 127.28.218.254:44755 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.28.218.254:44755: connect: Connection refused (error 111)
I20260806 22:50:17.054494 4957 raft_consensus.cc:482] T 00000000000000000000000000000000 P eb135056735a446dbff018e4072cab0c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 20186b499bd34a7d9d497ba785db0fc9)
I20260806 22:50:17.054625 4957 raft_consensus.cc:504] T 00000000000000000000000000000000 P eb135056735a446dbff018e4072cab0c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "20186b499bd34a7d9d497ba785db0fc9" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 44755 } } peers { permanent_uuid: "eb135056735a446dbff018e4072cab0c" member_type: VOTER last_known_addr { host: "127.28.218.253" port: 39243 } attrs { promote: false } } peers { permanent_uuid: "ed70ecfd24614dbc990e056d21844d64" member_type: VOTER last_known_addr { host: "127.28.218.252" port: 39503 } attrs { promote: false } }
I20260806 22:50:17.054911 4958 raft_consensus.cc:482] T 00000000000000000000000000000000 P ed70ecfd24614dbc990e056d21844d64 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 20186b499bd34a7d9d497ba785db0fc9)
I20260806 22:50:17.054973 4957 leader_election.cc:288] T 00000000000000000000000000000000 P eb135056735a446dbff018e4072cab0c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 20186b499bd34a7d9d497ba785db0fc9 (127.28.218.254:44755), ed70ecfd24614dbc990e056d21844d64 (127.28.218.252:39503)
I20260806 22:50:17.055106 4958 raft_consensus.cc:504] T 00000000000000000000000000000000 P ed70ecfd24614dbc990e056d21844d64 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "20186b499bd34a7d9d497ba785db0fc9" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 44755 } } peers { permanent_uuid: "eb135056735a446dbff018e4072cab0c" member_type: VOTER last_known_addr { host: "127.28.218.253" port: 39243 } attrs { promote: false } } peers { permanent_uuid: "ed70ecfd24614dbc990e056d21844d64" member_type: VOTER last_known_addr { host: "127.28.218.252" port: 39503 } attrs { promote: false } }
I20260806 22:50:17.055290 4959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:17.055052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:44755 (local address 127.0.0.1:52362)
0806 22:50:17.055197 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:17.055244 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:44755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":71,"threads_started":1}
W20260806 22:50:17.055430 4174 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.28.218.254:44755: connect: Connection refused (error 111)
I20260806 22:50:17.055783 4964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:17.055533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:44755 (local address 127.0.0.1:52364)
0806 22:50:17.055683 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:17.055737 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:44755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":62,"threads_started":1}
I20260806 22:50:17.055845 4959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:17.055741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:44755 (local address 127.0.0.1:52368)
0806 22:50:17.055780 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:17.055807 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:44755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260806 22:50:17.055931 4174 leader_election.cc:334] T 00000000000000000000000000000000 P eb135056735a446dbff018e4072cab0c [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 20186b499bd34a7d9d497ba785db0fc9 (127.28.218.254:44755): Network error: Client connection negotiation failed: client connection to 127.28.218.254:44755: connect: Connection refused (error 111)
W20260806 22:50:17.056216 4383 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.28.218.254:44755: connect: Connection refused (error 111)
I20260806 22:50:17.056402 4958 leader_election.cc:288] T 00000000000000000000000000000000 P ed70ecfd24614dbc990e056d21844d64 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 20186b499bd34a7d9d497ba785db0fc9 (127.28.218.254:44755), eb135056735a446dbff018e4072cab0c (127.28.218.253:39243)
I20260806 22:50:17.056643 4965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:17.056532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:44755 (local address 127.0.0.1:52380)
0806 22:50:17.056564 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:17.056605 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:44755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
W20260806 22:50:17.056717 4383 leader_election.cc:334] T 00000000000000000000000000000000 P ed70ecfd24614dbc990e056d21844d64 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 20186b499bd34a7d9d497ba785db0fc9 (127.28.218.254:44755): Network error: Client connection negotiation failed: client connection to 127.28.218.254:44755: connect: Connection refused (error 111)
I20260806 22:50:17.058552 4961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:17.055272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46014 (local address 127.28.218.252:39503)
0806 22:50:17.055483 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:17.055486 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:17.055495 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:50:17.055522 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:17.055522 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:17.055536 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:17.055606 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:50:17.055791 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:17.056562 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:17.057731 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:17.058281 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:17.058332 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:17.058339 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:17.058482 (+ 143us) server_negotiation.cc:299] Negotiation successful
0806 22:50:17.058512 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":192,"thread_start_us":42,"threads_started":1}
I20260806 22:50:17.058581 4960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:17.055149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.252:39503 (local address 127.0.0.1:46014)
0806 22:50:17.055338 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:17.055350 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:17.055407 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:17.055627 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:17.055628 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:17.055630 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:50:17.055701 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:17.055703 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:17.056635 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:17.056636 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:17.057649 (+ 1013us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:17.057652 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:17.058348 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:17.058349 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:17.058411 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:17.058415 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:50:17.058473 (+ 58us) client_negotiation.cc:240] Negotiation successful
0806 22:50:17.058513 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":40,"threads_started":1}
I20260806 22:50:17.058774 4412 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "eb135056735a446dbff018e4072cab0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "ed70ecfd24614dbc990e056d21844d64" is_pre_election: true
I20260806 22:50:17.058913 4412 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ed70ecfd24614dbc990e056d21844d64 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb135056735a446dbff018e4072cab0c in term 1.
I20260806 22:50:17.059023 4964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:17.055777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.253:39243 (local address 127.0.0.1:46446)
0806 22:50:17.055842 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:17.055852 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:17.055896 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:17.056212 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:17.056213 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:17.056216 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:50:17.056264 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:17.056266 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:17.057196 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:17.057197 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:17.058096 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:17.058098 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:17.058783 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:17.058784 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:17.058845 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:17.058849 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:50:17.058909 (+ 60us) client_negotiation.cc:240] Negotiation successful
0806 22:50:17.058952 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57,"thread_start_us":37,"threads_started":1}
I20260806 22:50:17.059039 4966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:17.055943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46446 (local address 127.28.218.253:39243)
0806 22:50:17.056069 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:17.056071 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:17.056079 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:50:17.056099 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:17.056100 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:17.056108 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:17.056337 (+ 229us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:50:17.056375 (+ 38us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:17.057112 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:17.058179 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:17.058685 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:17.058780 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:17.058787 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:17.058922 (+ 135us) server_negotiation.cc:299] Negotiation successful
0806 22:50:17.058974 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":92,"threads_started":1}
I20260806 22:50:17.059056 4175 leader_election.cc:302] T 00000000000000000000000000000000 P eb135056735a446dbff018e4072cab0c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: eb135056735a446dbff018e4072cab0c, ed70ecfd24614dbc990e056d21844d64; no voters: 20186b499bd34a7d9d497ba785db0fc9
I20260806 22:50:17.059192 4957 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eb135056735a446dbff018e4072cab0c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260806 22:50:17.059243 4957 raft_consensus.cc:482] T 00000000000000000000000000000000 P eb135056735a446dbff018e4072cab0c [term 1 FOLLOWER]: Starting leader election (detected failure of leader 20186b499bd34a7d9d497ba785db0fc9)
I20260806 22:50:17.059262 4957 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eb135056735a446dbff018e4072cab0c [term 1 FOLLOWER]: Advancing to term 2
I20260806 22:50:17.059288 4200 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ed70ecfd24614dbc990e056d21844d64" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "eb135056735a446dbff018e4072cab0c" is_pre_election: true
I20260806 22:50:17.060006 4957 raft_consensus.cc:504] T 00000000000000000000000000000000 P eb135056735a446dbff018e4072cab0c [term 2 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "20186b499bd34a7d9d497ba785db0fc9" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 44755 } } peers { permanent_uuid: "eb135056735a446dbff018e4072cab0c" member_type: VOTER last_known_addr { host: "127.28.218.253" port: 39243 } attrs { promote: false } } peers { permanent_uuid: "ed70ecfd24614dbc990e056d21844d64" member_type: VOTER last_known_addr { host: "127.28.218.252" port: 39503 } attrs { promote: false } }
I20260806 22:50:17.060137 4200 raft_consensus.cc:2379] T 00000000000000000000000000000000 P eb135056735a446dbff018e4072cab0c [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ed70ecfd24614dbc990e056d21844d64 in current term 2: Already voted for candidate eb135056735a446dbff018e4072cab0c in this term.
I20260806 22:50:17.060170 4957 leader_election.cc:288] T 00000000000000000000000000000000 P eb135056735a446dbff018e4072cab0c [CANDIDATE]: Term 2 election: Requested vote from peers 20186b499bd34a7d9d497ba785db0fc9 (127.28.218.254:44755), ed70ecfd24614dbc990e056d21844d64 (127.28.218.252:39503)
I20260806 22:50:17.060328 4412 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "eb135056735a446dbff018e4072cab0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "ed70ecfd24614dbc990e056d21844d64"
I20260806 22:50:17.060410 4412 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ed70ecfd24614dbc990e056d21844d64 [term 1 FOLLOWER]: Advancing to term 2
I20260806 22:50:17.060413 4960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:17.060334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:44755 (local address 127.0.0.1:52382)
0806 22:50:17.060364 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:17.060389 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:44755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
I20260806 22:50:17.060467 4384 leader_election.cc:302] T 00000000000000000000000000000000 P ed70ecfd24614dbc990e056d21844d64 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ed70ecfd24614dbc990e056d21844d64; no voters: 20186b499bd34a7d9d497ba785db0fc9, eb135056735a446dbff018e4072cab0c
I20260806 22:50:17.060678 4960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:17.060607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:44755 (local address 127.0.0.1:52390)
0806 22:50:17.060639 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:17.060658 (+ 19us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:44755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
W20260806 22:50:17.060757 4174 leader_election.cc:334] T 00000000000000000000000000000000 P eb135056735a446dbff018e4072cab0c [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 20186b499bd34a7d9d497ba785db0fc9 (127.28.218.254:44755): Network error: Client connection negotiation failed: client connection to 127.28.218.254:44755: connect: Connection refused (error 111)
I20260806 22:50:17.061113 4412 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ed70ecfd24614dbc990e056d21844d64 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb135056735a446dbff018e4072cab0c in term 2.
I20260806 22:50:17.061235 4958 raft_consensus.cc:2726] T 00000000000000000000000000000000 P ed70ecfd24614dbc990e056d21844d64 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 22:50:17.061280 4175 leader_election.cc:302] T 00000000000000000000000000000000 P eb135056735a446dbff018e4072cab0c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: eb135056735a446dbff018e4072cab0c, ed70ecfd24614dbc990e056d21844d64; no voters: 20186b499bd34a7d9d497ba785db0fc9
I20260806 22:50:17.061430 4957 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eb135056735a446dbff018e4072cab0c [term 2 FOLLOWER]: Leader election won for term 2
I20260806 22:50:17.061573 4957 raft_consensus.cc:686] T 00000000000000000000000000000000 P eb135056735a446dbff018e4072cab0c [term 2 LEADER]: Becoming Leader. State: Replica: eb135056735a446dbff018e4072cab0c, State: Running, Role: LEADER
I20260806 22:50:17.061669 4957 consensus_queue.cc:237] T 00000000000000000000000000000000 P eb135056735a446dbff018e4072cab0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "20186b499bd34a7d9d497ba785db0fc9" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 44755 } } peers { permanent_uuid: "eb135056735a446dbff018e4072cab0c" member_type: VOTER last_known_addr { host: "127.28.218.253" port: 39243 } attrs { promote: false } } peers { permanent_uuid: "ed70ecfd24614dbc990e056d21844d64" member_type: VOTER last_known_addr { host: "127.28.218.252" port: 39503 } attrs { promote: false } }
I20260806 22:50:17.062007 4968 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb135056735a446dbff018e4072cab0c [sys.catalog]: SysCatalogTable state changed. Reason: New leader eb135056735a446dbff018e4072cab0c. Latest consensus state: current_term: 2 leader_uuid: "eb135056735a446dbff018e4072cab0c" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "20186b499bd34a7d9d497ba785db0fc9" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 44755 } } peers { permanent_uuid: "eb135056735a446dbff018e4072cab0c" member_type: VOTER last_known_addr { host: "127.28.218.253" port: 39243 } attrs { promote: false } } peers { permanent_uuid: "ed70ecfd24614dbc990e056d21844d64" member_type: VOTER last_known_addr { host: "127.28.218.252" port: 39503 } attrs { promote: false } } }
I20260806 22:50:17.062100 4968 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb135056735a446dbff018e4072cab0c [sys.catalog]: This master's current role is: LEADER
I20260806 22:50:17.062254 4970 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:50:17.062673 4970 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:50:17.062733 4970 catalog_manager.cc:1259] Loaded cluster ID: 49a945fc1aed48f391333d0beb7d3f21
I20260806 22:50:17.062749 4970 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:50:17.062863 4970 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:50:17.062914 4970 catalog_manager.cc:5879] T 00000000000000000000000000000000 P eb135056735a446dbff018e4072cab0c: Loaded TSK: 0
I20260806 22:50:17.062996 4970 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:50:17.111291 4190 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46444:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:50:17.112895 4190 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:50:17.113866 4960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:17.113756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:44755 (local address 127.0.0.1:52414)
0806 22:50:17.113809 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:17.113837 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:44755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260806 22:50:17.113924 4412 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ed70ecfd24614dbc990e056d21844d64 [term 2 FOLLOWER]: Refusing update from remote peer eb135056735a446dbff018e4072cab0c: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260806 22:50:17.114292 4960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:17.114142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:44755 (local address 127.0.0.1:52420)
0806 22:50:17.114192 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:17.114242 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:44755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260806 22:50:17.114235 4968 consensus_queue.cc:1035] T 00000000000000000000000000000000 P eb135056735a446dbff018e4072cab0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed70ecfd24614dbc990e056d21844d64" member_type: VOTER last_known_addr { host: "127.28.218.252" port: 39503 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260806 22:50:17.114389 4174 consensus_peers.cc:487] T 00000000000000000000000000000000 P eb135056735a446dbff018e4072cab0c -> Peer 20186b499bd34a7d9d497ba785db0fc9 (127.28.218.254:44755): Couldn't send request to peer 20186b499bd34a7d9d497ba785db0fc9. Status: Network error: Client connection negotiation failed: client connection to 127.28.218.254:44755: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260806 22:50:17.114840 4958 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed70ecfd24614dbc990e056d21844d64 [sys.catalog]: SysCatalogTable state changed. Reason: New leader eb135056735a446dbff018e4072cab0c. Latest consensus state: current_term: 2 leader_uuid: "eb135056735a446dbff018e4072cab0c" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "20186b499bd34a7d9d497ba785db0fc9" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 44755 } } peers { permanent_uuid: "eb135056735a446dbff018e4072cab0c" member_type: VOTER last_known_addr { host: "127.28.218.253" port: 39243 } attrs { promote: false } } peers { permanent_uuid: "ed70ecfd24614dbc990e056d21844d64" member_type: VOTER last_known_addr { host: "127.28.218.252" port: 39503 } attrs { promote: false } } }
I20260806 22:50:17.114953 4958 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed70ecfd24614dbc990e056d21844d64 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:50:17.115368 4971 mvcc.cc:205] Tried to move back new op lower bound from 7315687903696584704 to 7315687903485206528. Current Snapshot: MvccSnapshot[applied={T|T < 7315687903696584704}]
I20260806 22:50:17.115464 4957 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb135056735a446dbff018e4072cab0c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "eb135056735a446dbff018e4072cab0c" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "20186b499bd34a7d9d497ba785db0fc9" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 44755 } } peers { permanent_uuid: "eb135056735a446dbff018e4072cab0c" member_type: VOTER last_known_addr { host: "127.28.218.253" port: 39243 } attrs { promote: false } } peers { permanent_uuid: "ed70ecfd24614dbc990e056d21844d64" member_type: VOTER last_known_addr { host: "127.28.218.252" port: 39503 } attrs { promote: false } } }
I20260806 22:50:17.115563 4957 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb135056735a446dbff018e4072cab0c [sys.catalog]: This master's current role is: LEADER
I20260806 22:50:17.115669 4958 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed70ecfd24614dbc990e056d21844d64 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "eb135056735a446dbff018e4072cab0c" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "20186b499bd34a7d9d497ba785db0fc9" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 44755 } } peers { permanent_uuid: "eb135056735a446dbff018e4072cab0c" member_type: VOTER last_known_addr { host: "127.28.218.253" port: 39243 } attrs { promote: false } } peers { permanent_uuid: "ed70ecfd24614dbc990e056d21844d64" member_type: VOTER last_known_addr { host: "127.28.218.252" port: 39503 } attrs { promote: false } } }
I20260806 22:50:17.115748 4958 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed70ecfd24614dbc990e056d21844d64 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:50:17.120453 4960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:17.117332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.194:45405 (local address 127.0.0.1:45660)
0806 22:50:17.117360 (+ 28us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:17.117371 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:17.117410 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:17.117655 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:17.117656 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:17.117658 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:50:17.117711 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:17.117713 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:17.118548 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:17.118549 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:17.119443 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:17.119445 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:17.120254 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:17.120255 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:17.120310 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:17.120315 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:50:17.120343 (+ 28us) client_negotiation.cc:240] Negotiation successful
0806 22:50:17.120379 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15}
I20260806 22:50:17.120738 4977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:17.117408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45660 (local address 127.28.218.194:45405)
0806 22:50:17.117516 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:17.117518 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:17.117526 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:50:17.117553 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:17.117554 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:17.117565 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:17.117781 (+ 216us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:50:17.117827 (+ 46us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:17.118468 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:17.119542 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:17.120149 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:17.120585 (+ 436us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:17.120595 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:17.120614 (+ 19us) server_negotiation.cc:299] Negotiation successful
0806 22:50:17.120660 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89,"thread_start_us":52,"threads_started":1}
I20260806 22:50:17.120951 4668 tablet_service.cc:1467] Processing CreateTablet for tablet cd7981845a6a4fb8b3ec6f977422701a (DEFAULT_TABLE table=test-table [id=61164dc65c3a48418692a06eda893478]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:50:17.121102 4668 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cd7981845a6a4fb8b3ec6f977422701a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:50:17.121162 4978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:17.117441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.193:35729 (local address 127.0.0.1:60844)
0806 22:50:17.117815 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:17.117825 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:17.117875 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:17.118049 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:17.118050 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:17.118052 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:50:17.118090 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:17.118091 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:17.119107 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:17.119108 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:17.120126 (+ 1018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:17.120128 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:17.120920 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:17.120921 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:17.120979 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:17.120983 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:50:17.121036 (+ 53us) client_negotiation.cc:240] Negotiation successful
0806 22:50:17.121081 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":356,"mutex_wait_us":2,"thread_start_us":82,"threads_started":1}
I20260806 22:50:17.121436 4959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:17.117422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.195:42091 (local address 127.0.0.1:43168)
0806 22:50:17.117465 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:17.117474 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:17.117522 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:17.117983 (+ 461us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:17.117984 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:17.117986 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:50:17.118029 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:17.118031 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:17.119064 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:17.119065 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:17.120154 (+ 1089us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:17.120157 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:17.121204 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:17.121205 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:17.121278 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:17.121281 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:50:17.121328 (+ 47us) client_negotiation.cc:240] Negotiation successful
0806 22:50:17.121367 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28,"mutex_wait_us":1}
I20260806 22:50:17.121443 4980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:17.117627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60844 (local address 127.28.218.193:35729)
0806 22:50:17.117771 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:17.117773 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:17.117871 (+ 98us) server_negotiation.cc:407] Connection header received
0806 22:50:17.117929 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:17.117930 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:17.117942 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:17.118000 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:50:17.118174 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:17.119028 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:17.120209 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:17.120737 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:17.120819 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:17.120828 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:17.121334 (+ 506us) server_negotiation.cc:299] Negotiation successful
0806 22:50:17.121380 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":73,"threads_started":1}
I20260806 22:50:17.121678 4979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:17.117545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43168 (local address 127.28.218.195:42091)
0806 22:50:17.117643 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:17.117645 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:17.117652 (+ 7us) server_negotiation.cc:407] Connection header received
0806 22:50:17.117685 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:17.117686 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:17.117694 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:17.118103 (+ 409us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:50:17.118174 (+ 71us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:17.118960 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:17.120257 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:17.121092 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:17.121168 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:17.121176 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:17.121550 (+ 374us) server_negotiation.cc:299] Negotiation successful
0806 22:50:17.121613 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81,"thread_start_us":50,"threads_started":1}
I20260806 22:50:17.122463 4981 tablet_bootstrap.cc:492] T cd7981845a6a4fb8b3ec6f977422701a P 05b84cde108549378d6584ef921fed23: Bootstrap starting.
I20260806 22:50:17.122627 4522 tablet_service.cc:1467] Processing CreateTablet for tablet cd7981845a6a4fb8b3ec6f977422701a (DEFAULT_TABLE table=test-table [id=61164dc65c3a48418692a06eda893478]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:50:17.122772 4522 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cd7981845a6a4fb8b3ec6f977422701a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:50:17.122931 4981 tablet_bootstrap.cc:654] T cd7981845a6a4fb8b3ec6f977422701a P 05b84cde108549378d6584ef921fed23: Neither blocks nor log segments found. Creating new log.
I20260806 22:50:17.123306 4814 tablet_service.cc:1467] Processing CreateTablet for tablet cd7981845a6a4fb8b3ec6f977422701a (DEFAULT_TABLE table=test-table [id=61164dc65c3a48418692a06eda893478]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:50:17.123445 4814 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cd7981845a6a4fb8b3ec6f977422701a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:50:17.124229 4983 tablet_bootstrap.cc:492] T cd7981845a6a4fb8b3ec6f977422701a P 7785c46d26114840970c8f163edf0196: Bootstrap starting.
I20260806 22:50:17.124624 4983 tablet_bootstrap.cc:654] T cd7981845a6a4fb8b3ec6f977422701a P 7785c46d26114840970c8f163edf0196: Neither blocks nor log segments found. Creating new log.
I20260806 22:50:17.124689 4986 tablet_bootstrap.cc:492] T cd7981845a6a4fb8b3ec6f977422701a P b11173b0207c46549ca94b7a987905cf: Bootstrap starting.
I20260806 22:50:17.124848 4981 tablet_bootstrap.cc:492] T cd7981845a6a4fb8b3ec6f977422701a P 05b84cde108549378d6584ef921fed23: No bootstrap required, opened a new log
I20260806 22:50:17.124908 4981 ts_tablet_manager.cc:1397] T cd7981845a6a4fb8b3ec6f977422701a P 05b84cde108549378d6584ef921fed23: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:50:17.125092 4986 tablet_bootstrap.cc:654] T cd7981845a6a4fb8b3ec6f977422701a P b11173b0207c46549ca94b7a987905cf: Neither blocks nor log segments found. Creating new log.
I20260806 22:50:17.125069 4981 raft_consensus.cc:348] T cd7981845a6a4fb8b3ec6f977422701a P 05b84cde108549378d6584ef921fed23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b84cde108549378d6584ef921fed23" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 45405 } } peers { permanent_uuid: "7785c46d26114840970c8f163edf0196" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 35729 } } peers { permanent_uuid: "b11173b0207c46549ca94b7a987905cf" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 42091 } }
I20260806 22:50:17.125141 4981 raft_consensus.cc:374] T cd7981845a6a4fb8b3ec6f977422701a P 05b84cde108549378d6584ef921fed23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:50:17.125165 4981 raft_consensus.cc:729] T cd7981845a6a4fb8b3ec6f977422701a P 05b84cde108549378d6584ef921fed23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05b84cde108549378d6584ef921fed23, State: Initialized, Role: FOLLOWER
I20260806 22:50:17.125219 4981 consensus_queue.cc:260] T cd7981845a6a4fb8b3ec6f977422701a P 05b84cde108549378d6584ef921fed23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b84cde108549378d6584ef921fed23" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 45405 } } peers { permanent_uuid: "7785c46d26114840970c8f163edf0196" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 35729 } } peers { permanent_uuid: "b11173b0207c46549ca94b7a987905cf" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 42091 } }
I20260806 22:50:17.125380 4981 ts_tablet_manager.cc:1428] T cd7981845a6a4fb8b3ec6f977422701a P 05b84cde108549378d6584ef921fed23: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260806 22:50:17.125622 4983 tablet_bootstrap.cc:492] T cd7981845a6a4fb8b3ec6f977422701a P 7785c46d26114840970c8f163edf0196: No bootstrap required, opened a new log
I20260806 22:50:17.125685 4983 ts_tablet_manager.cc:1397] T cd7981845a6a4fb8b3ec6f977422701a P 7785c46d26114840970c8f163edf0196: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 22:50:17.125886 4986 tablet_bootstrap.cc:492] T cd7981845a6a4fb8b3ec6f977422701a P b11173b0207c46549ca94b7a987905cf: No bootstrap required, opened a new log
I20260806 22:50:17.125865 4983 raft_consensus.cc:348] T cd7981845a6a4fb8b3ec6f977422701a P 7785c46d26114840970c8f163edf0196 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b84cde108549378d6584ef921fed23" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 45405 } } peers { permanent_uuid: "7785c46d26114840970c8f163edf0196" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 35729 } } peers { permanent_uuid: "b11173b0207c46549ca94b7a987905cf" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 42091 } }
I20260806 22:50:17.125931 4983 raft_consensus.cc:374] T cd7981845a6a4fb8b3ec6f977422701a P 7785c46d26114840970c8f163edf0196 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:50:17.125932 4986 ts_tablet_manager.cc:1397] T cd7981845a6a4fb8b3ec6f977422701a P b11173b0207c46549ca94b7a987905cf: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 22:50:17.125953 4983 raft_consensus.cc:729] T cd7981845a6a4fb8b3ec6f977422701a P 7785c46d26114840970c8f163edf0196 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7785c46d26114840970c8f163edf0196, State: Initialized, Role: FOLLOWER
I20260806 22:50:17.126032 4986 raft_consensus.cc:348] T cd7981845a6a4fb8b3ec6f977422701a P b11173b0207c46549ca94b7a987905cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b84cde108549378d6584ef921fed23" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 45405 } } peers { permanent_uuid: "7785c46d26114840970c8f163edf0196" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 35729 } } peers { permanent_uuid: "b11173b0207c46549ca94b7a987905cf" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 42091 } }
I20260806 22:50:17.126097 4986 raft_consensus.cc:374] T cd7981845a6a4fb8b3ec6f977422701a P b11173b0207c46549ca94b7a987905cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:50:17.126125 4986 raft_consensus.cc:729] T cd7981845a6a4fb8b3ec6f977422701a P b11173b0207c46549ca94b7a987905cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b11173b0207c46549ca94b7a987905cf, State: Initialized, Role: FOLLOWER
I20260806 22:50:17.126178 4986 consensus_queue.cc:260] T cd7981845a6a4fb8b3ec6f977422701a P b11173b0207c46549ca94b7a987905cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b84cde108549378d6584ef921fed23" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 45405 } } peers { permanent_uuid: "7785c46d26114840970c8f163edf0196" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 35729 } } peers { permanent_uuid: "b11173b0207c46549ca94b7a987905cf" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 42091 } }
I20260806 22:50:17.126231 4983 consensus_queue.cc:260] T cd7981845a6a4fb8b3ec6f977422701a P 7785c46d26114840970c8f163edf0196 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b84cde108549378d6584ef921fed23" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 45405 } } peers { permanent_uuid: "7785c46d26114840970c8f163edf0196" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 35729 } } peers { permanent_uuid: "b11173b0207c46549ca94b7a987905cf" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 42091 } }
I20260806 22:50:17.126358 4986 ts_tablet_manager.cc:1428] T cd7981845a6a4fb8b3ec6f977422701a P b11173b0207c46549ca94b7a987905cf: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260806 22:50:17.126443 4983 ts_tablet_manager.cc:1428] T cd7981845a6a4fb8b3ec6f977422701a P 7785c46d26114840970c8f163edf0196: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 22:50:17.126902 4995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:17.126689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:44755 (local address 127.28.218.195:58215)
0806 22:50:17.126817 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:17.126866 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:44755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":67,"threads_started":1}
I20260806 22:50:17.126958 4990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:17.125610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:44755 (local address 127.28.218.194:52435)
0806 22:50:17.126893 (+ 1283us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:17.126926 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:44755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1264,"thread_start_us":86,"threads_started":1}
I20260806 22:50:17.127188 4993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:17.126665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:44755 (local address 127.28.218.193:56245)
0806 22:50:17.127075 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:17.127114 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:44755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":77,"threads_started":1}
I20260806 22:50:17.127326 4990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:17.127248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:44755 (local address 127.28.218.194:50327)
0806 22:50:17.127280 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:17.127301 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:44755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
I20260806 22:50:17.127497 4742 heartbeater.cc:503] Master 127.28.218.253:39243 was elected leader, sending a full tablet report...
I20260806 22:50:17.127576 4993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:17.127385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:44755 (local address 127.28.218.193:60203)
0806 22:50:17.127517 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:17.127544 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:44755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":97,"threads_started":1}
I20260806 22:50:17.127813 4888 heartbeater.cc:503] Master 127.28.218.253:39243 was elected leader, sending a full tablet report...
I20260806 22:50:17.127998 4995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:17.127916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:44755 (local address 127.28.218.195:43281)
0806 22:50:17.127947 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:17.127972 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:44755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
W20260806 22:50:17.153329 4890 tablet.cc:2369] T cd7981845a6a4fb8b3ec6f977422701a P b11173b0207c46549ca94b7a987905cf: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 22:50:17.375017 4598 tablet.cc:2369] T cd7981845a6a4fb8b3ec6f977422701a P 7785c46d26114840970c8f163edf0196: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:50:17.464363 4991 raft_consensus.cc:482] T cd7981845a6a4fb8b3ec6f977422701a P b11173b0207c46549ca94b7a987905cf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:50:17.464478 4991 raft_consensus.cc:504] T cd7981845a6a4fb8b3ec6f977422701a P b11173b0207c46549ca94b7a987905cf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b84cde108549378d6584ef921fed23" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 45405 } } peers { permanent_uuid: "7785c46d26114840970c8f163edf0196" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 35729 } } peers { permanent_uuid: "b11173b0207c46549ca94b7a987905cf" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 42091 } }
I20260806 22:50:17.464627 4991 leader_election.cc:288] T cd7981845a6a4fb8b3ec6f977422701a P b11173b0207c46549ca94b7a987905cf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 05b84cde108549378d6584ef921fed23 (127.28.218.194:45405), 7785c46d26114840970c8f163edf0196 (127.28.218.193:35729)
I20260806 22:50:17.464931 4691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd7981845a6a4fb8b3ec6f977422701a" candidate_uuid: "b11173b0207c46549ca94b7a987905cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05b84cde108549378d6584ef921fed23" is_pre_election: true
I20260806 22:50:17.465046 4691 raft_consensus.cc:2454] T cd7981845a6a4fb8b3ec6f977422701a P 05b84cde108549378d6584ef921fed23 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b11173b0207c46549ca94b7a987905cf in term 0.
I20260806 22:50:17.464996 4543 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd7981845a6a4fb8b3ec6f977422701a" candidate_uuid: "b11173b0207c46549ca94b7a987905cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7785c46d26114840970c8f163edf0196" is_pre_election: true
I20260806 22:50:17.465127 4543 raft_consensus.cc:2454] T cd7981845a6a4fb8b3ec6f977422701a P 7785c46d26114840970c8f163edf0196 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b11173b0207c46549ca94b7a987905cf in term 0.
I20260806 22:50:17.465253 4763 leader_election.cc:302] T cd7981845a6a4fb8b3ec6f977422701a P b11173b0207c46549ca94b7a987905cf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05b84cde108549378d6584ef921fed23, b11173b0207c46549ca94b7a987905cf; no voters:
I20260806 22:50:17.465365 4991 raft_consensus.cc:2781] T cd7981845a6a4fb8b3ec6f977422701a P b11173b0207c46549ca94b7a987905cf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:50:17.465432 4991 raft_consensus.cc:482] T cd7981845a6a4fb8b3ec6f977422701a P b11173b0207c46549ca94b7a987905cf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:50:17.465463 4991 raft_consensus.cc:3037] T cd7981845a6a4fb8b3ec6f977422701a P b11173b0207c46549ca94b7a987905cf [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:50:17.466310 4991 raft_consensus.cc:504] T cd7981845a6a4fb8b3ec6f977422701a P b11173b0207c46549ca94b7a987905cf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b84cde108549378d6584ef921fed23" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 45405 } } peers { permanent_uuid: "7785c46d26114840970c8f163edf0196" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 35729 } } peers { permanent_uuid: "b11173b0207c46549ca94b7a987905cf" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 42091 } }
I20260806 22:50:17.466460 4991 leader_election.cc:288] T cd7981845a6a4fb8b3ec6f977422701a P b11173b0207c46549ca94b7a987905cf [CANDIDATE]: Term 1 election: Requested vote from peers 05b84cde108549378d6584ef921fed23 (127.28.218.194:45405), 7785c46d26114840970c8f163edf0196 (127.28.218.193:35729)
I20260806 22:50:17.466629 4691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd7981845a6a4fb8b3ec6f977422701a" candidate_uuid: "b11173b0207c46549ca94b7a987905cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05b84cde108549378d6584ef921fed23"
I20260806 22:50:17.466650 4543 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd7981845a6a4fb8b3ec6f977422701a" candidate_uuid: "b11173b0207c46549ca94b7a987905cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7785c46d26114840970c8f163edf0196"
I20260806 22:50:17.466713 4691 raft_consensus.cc:3037] T cd7981845a6a4fb8b3ec6f977422701a P 05b84cde108549378d6584ef921fed23 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:50:17.466734 4543 raft_consensus.cc:3037] T cd7981845a6a4fb8b3ec6f977422701a P 7785c46d26114840970c8f163edf0196 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:50:17.467438 4691 raft_consensus.cc:2454] T cd7981845a6a4fb8b3ec6f977422701a P 05b84cde108549378d6584ef921fed23 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b11173b0207c46549ca94b7a987905cf in term 1.
I20260806 22:50:17.467504 4543 raft_consensus.cc:2454] T cd7981845a6a4fb8b3ec6f977422701a P 7785c46d26114840970c8f163edf0196 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b11173b0207c46549ca94b7a987905cf in term 1.
I20260806 22:50:17.467603 4763 leader_election.cc:302] T cd7981845a6a4fb8b3ec6f977422701a P b11173b0207c46549ca94b7a987905cf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05b84cde108549378d6584ef921fed23, b11173b0207c46549ca94b7a987905cf; no voters:
I20260806 22:50:17.467706 4991 raft_consensus.cc:2781] T cd7981845a6a4fb8b3ec6f977422701a P b11173b0207c46549ca94b7a987905cf [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:50:17.467928 4991 raft_consensus.cc:686] T cd7981845a6a4fb8b3ec6f977422701a P b11173b0207c46549ca94b7a987905cf [term 1 LEADER]: Becoming Leader. State: Replica: b11173b0207c46549ca94b7a987905cf, State: Running, Role: LEADER
I20260806 22:50:17.467990 4991 consensus_queue.cc:237] T cd7981845a6a4fb8b3ec6f977422701a P b11173b0207c46549ca94b7a987905cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b84cde108549378d6584ef921fed23" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 45405 } } peers { permanent_uuid: "7785c46d26114840970c8f163edf0196" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 35729 } } peers { permanent_uuid: "b11173b0207c46549ca94b7a987905cf" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 42091 } }
I20260806 22:50:17.468381 4995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:17.468262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:44755 (local address 127.28.218.195:58727)
0806 22:50:17.468314 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:17.468343 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:44755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260806 22:50:17.468632 4190 catalog_manager.cc:5495] T cd7981845a6a4fb8b3ec6f977422701a P b11173b0207c46549ca94b7a987905cf reported cstate change: term changed from 0 to 1, leader changed from <none> to b11173b0207c46549ca94b7a987905cf (127.28.218.195). New cstate: current_term: 1 leader_uuid: "b11173b0207c46549ca94b7a987905cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b84cde108549378d6584ef921fed23" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 45405 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7785c46d26114840970c8f163edf0196" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 35729 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b11173b0207c46549ca94b7a987905cf" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 42091 } health_report { overall_health: HEALTHY } } }
I20260806 22:50:17.519156 4978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:17.518992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:44755 (local address 127.0.0.1:52428)
0806 22:50:17.519048 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:17.519095 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:44755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260806 22:50:17.519698 4978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:17.519596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:44755 (local address 127.0.0.1:52434)
0806 22:50:17.519642 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:17.519673 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:44755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260806 22:50:17.561146 4979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:17.558726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43174 (local address 127.28.218.195:42091)
0806 22:50:17.558802 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:17.558804 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:17.558813 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:50:17.558915 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:17.558916 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:17.558928 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:17.558976 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:50:17.559118 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:17.559666 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:17.560431 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:17.560654 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:17.560729 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:17.560884 (+ 155us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:50:17.560915 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:50:17.560982 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:17.561041 (+ 59us) server_negotiation.cc:299] Negotiation successful
0806 22:50:17.561062 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 22:50:17.562073 4543 raft_consensus.cc:1261] T cd7981845a6a4fb8b3ec6f977422701a P 7785c46d26114840970c8f163edf0196 [term 1 FOLLOWER]: Refusing update from remote peer b11173b0207c46549ca94b7a987905cf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:50:17.562103 4691 raft_consensus.cc:1261] T cd7981845a6a4fb8b3ec6f977422701a P 05b84cde108549378d6584ef921fed23 [term 1 FOLLOWER]: Refusing update from remote peer b11173b0207c46549ca94b7a987905cf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:50:17.562335 4991 consensus_queue.cc:1035] T cd7981845a6a4fb8b3ec6f977422701a P b11173b0207c46549ca94b7a987905cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "7785c46d26114840970c8f163edf0196" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 35729 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:50:17.562459 4997 consensus_queue.cc:1035] T cd7981845a6a4fb8b3ec6f977422701a P b11173b0207c46549ca94b7a987905cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "05b84cde108549378d6584ef921fed23" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 45405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:50:17.562934 4597 heartbeater.cc:503] Master 127.28.218.253:39243 was elected leader, sending a full tablet report...
I20260806 22:50:17.563441 4996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:17.563185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:44755 (local address 127.28.218.193:58959)
0806 22:50:17.563348 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:17.563397 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:44755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135}
I20260806 22:50:17.563838 4990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:17.563705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:44755 (local address 127.28.218.194:48827)
0806 22:50:17.563765 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:17.563796 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:44755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260806 22:50:17.564558 4995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:17.564441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:44755 (local address 127.28.218.195:58611)
0806 22:50:17.564495 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:17.564528 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:44755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260806 22:50:17.564949 4990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:17.564866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:44755 (local address 127.28.218.194:43351)
0806 22:50:17.564901 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:17.564924 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:44755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260806 22:50:17.565070 5003 mvcc.cc:205] Tried to move back new op lower bound from 7315687905532506112 to 7315687905149399040. Current Snapshot: MvccSnapshot[applied={T|T < 7315687905532506112}]
I20260806 22:50:17.565867 4190 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46444:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:50:17.565958 4190 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46444:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:50:17.566107 4996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:17.566027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:44755 (local address 127.28.218.193:50757)
0806 22:50:17.566064 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:17.566086 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:44755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260806 22:50:17.567209 4190 catalog_manager.cc:5782] T 00000000000000000000000000000000 P eb135056735a446dbff018e4072cab0c: Sending DeleteTablet for 3 replicas of tablet cd7981845a6a4fb8b3ec6f977422701a
I20260806 22:50:17.567400 4522 tablet_service.cc:1514] Processing DeleteTablet for tablet cd7981845a6a4fb8b3ec6f977422701a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:50:17 UTC) from {username='oryx', principal='oryx/127.28.218.253@KRBTEST.COM'} at 127.0.0.1:60844
I20260806 22:50:17.567454 4521 tablet_service.cc:1514] Processing DeleteTablet for tablet cd7981845a6a4fb8b3ec6f977422701a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:50:17 UTC) from {username='oryx', principal='oryx/127.28.218.253@KRBTEST.COM'} at 127.0.0.1:60844
I20260806 22:50:17.567619 4814 tablet_service.cc:1514] Processing DeleteTablet for tablet cd7981845a6a4fb8b3ec6f977422701a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:50:17 UTC) from {username='oryx', principal='oryx/127.28.218.253@KRBTEST.COM'} at 127.0.0.1:43168
I20260806 22:50:17.567620 4668 tablet_service.cc:1514] Processing DeleteTablet for tablet cd7981845a6a4fb8b3ec6f977422701a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:50:17 UTC) from {username='oryx', principal='oryx/127.28.218.253@KRBTEST.COM'} at 127.0.0.1:45660
I20260806 22:50:17.567646 5009 tablet_replica.cc:330] T cd7981845a6a4fb8b3ec6f977422701a P 7785c46d26114840970c8f163edf0196: stopping tablet replica
I20260806 22:50:17.567700 5009 raft_consensus.cc:2229] T cd7981845a6a4fb8b3ec6f977422701a P 7785c46d26114840970c8f163edf0196 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:50:17.567746 5009 raft_consensus.cc:2258] T cd7981845a6a4fb8b3ec6f977422701a P 7785c46d26114840970c8f163edf0196 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:50:17.567919 5011 tablet_replica.cc:330] T cd7981845a6a4fb8b3ec6f977422701a P b11173b0207c46549ca94b7a987905cf: stopping tablet replica
I20260806 22:50:17.567996 5011 raft_consensus.cc:2229] T cd7981845a6a4fb8b3ec6f977422701a P b11173b0207c46549ca94b7a987905cf [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:50:17.568045 5011 raft_consensus.cc:2258] T cd7981845a6a4fb8b3ec6f977422701a P b11173b0207c46549ca94b7a987905cf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:50:17.568068 5009 ts_tablet_manager.cc:1905] T cd7981845a6a4fb8b3ec6f977422701a P 7785c46d26114840970c8f163edf0196: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:50:17.568151 5012 tablet_replica.cc:330] T cd7981845a6a4fb8b3ec6f977422701a P 05b84cde108549378d6584ef921fed23: stopping tablet replica
I20260806 22:50:17.568209 5012 raft_consensus.cc:2229] T cd7981845a6a4fb8b3ec6f977422701a P 05b84cde108549378d6584ef921fed23 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:50:17.568249 5012 raft_consensus.cc:2258] T cd7981845a6a4fb8b3ec6f977422701a P 05b84cde108549378d6584ef921fed23 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:50:17.568236 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 4349
I20260806 22:50:17.569676 5012 ts_tablet_manager.cc:1905] T cd7981845a6a4fb8b3ec6f977422701a P 05b84cde108549378d6584ef921fed23: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:50:17.571619 5012 ts_tablet_manager.cc:1918] T cd7981845a6a4fb8b3ec6f977422701a P 05b84cde108549378d6584ef921fed23: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:50:17.571677 5012 log.cc:1196] T cd7981845a6a4fb8b3ec6f977422701a P 05b84cde108549378d6584ef921fed23: Deleting WAL directory at /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/ts-1/wal/wals/cd7981845a6a4fb8b3ec6f977422701a
I20260806 22:50:17.571899 5012 ts_tablet_manager.cc:1939] T cd7981845a6a4fb8b3ec6f977422701a P 05b84cde108549378d6584ef921fed23: Deleting consensus metadata
I20260806 22:50:17.572242 4173 catalog_manager.cc:4841] TS 05b84cde108549378d6584ef921fed23 (127.28.218.194:45405): tablet cd7981845a6a4fb8b3ec6f977422701a (table test-table [id=61164dc65c3a48418692a06eda893478]) successfully deleted
W20260806 22:50:17.575657 4978 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:50:17.575520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.193:35729 (local address 127.0.0.1:60848)
0806 22:50:17.575577 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:17.575607 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.193:35729: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":35}
W20260806 22:50:17.575824 4173 catalog_manager.cc:4584] TS 7785c46d26114840970c8f163edf0196 (127.28.218.193:35729): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet cd7981845a6a4fb8b3ec6f977422701a: Network error: Client connection negotiation failed: client connection to 127.28.218.193:35729: connect: Connection reset by peer (error 104)
I20260806 22:50:17.575911 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 4603
I20260806 22:50:17.576033 5011 ts_tablet_manager.cc:1905] T cd7981845a6a4fb8b3ec6f977422701a P b11173b0207c46549ca94b7a987905cf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:50:17.577747 5011 ts_tablet_manager.cc:1918] T cd7981845a6a4fb8b3ec6f977422701a P b11173b0207c46549ca94b7a987905cf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:50:17.577788 5011 log.cc:1196] T cd7981845a6a4fb8b3ec6f977422701a P b11173b0207c46549ca94b7a987905cf: Deleting WAL directory at /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056586788339-29547-0/minicluster-data/ts-2/wal/wals/cd7981845a6a4fb8b3ec6f977422701a
I20260806 22:50:17.577982 5011 ts_tablet_manager.cc:1939] T cd7981845a6a4fb8b3ec6f977422701a P b11173b0207c46549ca94b7a987905cf: Deleting consensus metadata
I20260806 22:50:17.578222 4176 catalog_manager.cc:4841] TS b11173b0207c46549ca94b7a987905cf (127.28.218.195:42091): tablet cd7981845a6a4fb8b3ec6f977422701a (table test-table [id=61164dc65c3a48418692a06eda893478]) successfully deleted
I20260806 22:50:17.583357 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 4747
I20260806 22:50:17.590216 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 4050
I20260806 22:50:17.596005 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 4247
2026-08-06T22:50:17Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (5044 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056586788339-29547-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:50:17 dist-test-slave-1xrh krb5kdc[5015](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:50:17 dist-test-slave-1xrh krb5kdc[5015](info): set up 2 sockets
Aug 06 22:50:17 dist-test-slave-1xrh krb5kdc[5015](info): commencing operation
krb5kdc: starting...
W20260806 22:50:18.668370 29547 mini_kdc.cc:121] Time spent starting KDC: real 1.050s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056586788339-29547-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056586788339-29547-0/krb5kdc/test-user.keytab.
Aug 06 22:50:18 dist-test-slave-1xrh krb5kdc[5015](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056618, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:50:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:50:18Z Disabled control of system clock
WARNING: no policy specified for oryx/127.28.218.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.28.218.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.28.218.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056586788339-29547-0/krb5kdc/oryx_127.28.218.254.keytab.
Entry for principal oryx/127.28.218.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056586788339-29547-0/krb5kdc/oryx_127.28.218.254.keytab.
I20260806 22:50:18.784111 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056586788339-29547-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056586788339-29547-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056586788339-29547-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.218.254:33353
--webserver_interface=127.28.218.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056586788339-29547-0/krb5kdc/oryx_127.28.218.254.keytab
--principal=oryx/127.28.218.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.218.212:38031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.218.254:33353
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:50:18.865015 5028 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:50:18.865243 5028 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:50:18.865278 5028 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:50:18.866426 5028 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:50:18.866488 5028 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 22:50:18.866510 5028 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:50:18.866547 5028 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:50:18.866595 5028 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:50:18.866624 5028 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:50:18.868196 5028 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:38031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056586788339-29547-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056586788339-29547-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.218.254:33353
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056586788339-29547-0/krb5kdc/oryx_127.28.218.254.keytab
--principal=oryx/127.28.218.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.218.254:33353
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.218.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5028
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056586788339-29547-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:50:18.868413 5028 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:50:18.868571 5028 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:50:18.873052 5033 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:18.873039 5034 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:18.873116 5036 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:50:18.873260 5028 server_base.cc:1034] running on GCE node
I20260806 22:50:18.873504 5028 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:50:18.873723 5028 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:50:18.874864 5028 hybrid_clock.cc:648] HybridClock initialized: now 1786056618874840 us; error 39 us; skew 500 ppm
Aug 06 22:50:18 dist-test-slave-1xrh krb5kdc[5015](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056618, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.218.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:50:18.877324 5028 init.cc:377] Logged in from keytab as oryx/127.28.218.254@KRBTEST.COM (short username oryx)
I20260806 22:50:18.877785 5028 webserver.cc:460] Webserver started at http://127.28.218.254:43429/ using document root <none> and password file <none>
I20260806 22:50:18.878034 5028 fs_manager.cc:359] Metadata directory not provided
I20260806 22:50:18.878103 5028 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:50:18.878247 5028 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:50:18.879375 5028 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056586788339-29547-0/minicluster-data/master-0/data/instance:
uuid: "ecdca47ce9f544afa6f7395b0ad3445e"
format_stamp: "Formatted at 2026-08-06 22:50:18 on dist-test-slave-1xrh"
server_key: "087efdcce59d217c9c891af1c555f8d5"
server_key_iv: "bcfba36b3f5e8e948c9758904a4ab490"
server_key_version: "encryptionkey@0"
I20260806 22:50:18.879725 5028 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056586788339-29547-0/minicluster-data/master-0/wal/instance:
uuid: "ecdca47ce9f544afa6f7395b0ad3445e"
format_stamp: "Formatted at 2026-08-06 22:50:18 on dist-test-slave-1xrh"
server_key: "087efdcce59d217c9c891af1c555f8d5"
server_key_iv: "bcfba36b3f5e8e948c9758904a4ab490"
server_key_version: "encryptionkey@0"
I20260806 22:50:18.881053 5028 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:50:18.881757 5043 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:18.881942 5028 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:50:18.882040 5028 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056586788339-29547-0/minicluster-data/master-0/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056586788339-29547-0/minicluster-data/master-0/wal
uuid: "ecdca47ce9f544afa6f7395b0ad3445e"
format_stamp: "Formatted at 2026-08-06 22:50:18 on dist-test-slave-1xrh"
server_key: "087efdcce59d217c9c891af1c555f8d5"
server_key_iv: "bcfba36b3f5e8e948c9758904a4ab490"
server_key_version: "encryptionkey@0"
I20260806 22:50:18.882117 5028 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056586788339-29547-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056586788339-29547-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056586788339-29547-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:50:18.906270 5028 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:50:18.908077 5028 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:50:18.908232 5028 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:50:18.912895 5028 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.254:33353
I20260806 22:50:18.912938 5116 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.254:33353 every 8 connection(s)
I20260806 22:50:18.913250 5028 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
I20260806 22:50:18.913868 5117 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:50:18.916105 5117 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ecdca47ce9f544afa6f7395b0ad3445e: Bootstrap starting.
I20260806 22:50:18.916679 5117 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ecdca47ce9f544afa6f7395b0ad3445e: Neither blocks nor log segments found. Creating new log.
I20260806 22:50:18.916993 5117 log.cc:824] T 00000000000000000000000000000000 P ecdca47ce9f544afa6f7395b0ad3445e: Log is configured to *not* fsync() on all Append() calls
I20260806 22:50:18.917635 5117 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ecdca47ce9f544afa6f7395b0ad3445e: No bootstrap required, opened a new log
I20260806 22:50:18.918972 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 5028
I20260806 22:50:18.919080 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056586788339-29547-0/minicluster-data/master-0/wal/instance
I20260806 22:50:18.919350 29547 external_mini_cluster.cc:1428] Setting key 2254d7e6cfb70b56b6a330dbef7fd2ff
I20260806 22:50:18.920101 5117 raft_consensus.cc:348] T 00000000000000000000000000000000 P ecdca47ce9f544afa6f7395b0ad3445e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecdca47ce9f544afa6f7395b0ad3445e" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 33353 } }
I20260806 22:50:18.920230 5117 raft_consensus.cc:374] T 00000000000000000000000000000000 P ecdca47ce9f544afa6f7395b0ad3445e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:50:18.920251 5117 raft_consensus.cc:729] T 00000000000000000000000000000000 P ecdca47ce9f544afa6f7395b0ad3445e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ecdca47ce9f544afa6f7395b0ad3445e, State: Initialized, Role: FOLLOWER
I20260806 22:50:18.920373 5117 consensus_queue.cc:260] T 00000000000000000000000000000000 P ecdca47ce9f544afa6f7395b0ad3445e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecdca47ce9f544afa6f7395b0ad3445e" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 33353 } }
I20260806 22:50:18.920462 5117 raft_consensus.cc:388] T 00000000000000000000000000000000 P ecdca47ce9f544afa6f7395b0ad3445e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:50:18.920506 5117 raft_consensus.cc:482] T 00000000000000000000000000000000 P ecdca47ce9f544afa6f7395b0ad3445e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:50:18.920557 5117 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ecdca47ce9f544afa6f7395b0ad3445e [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:50:18.921085 5117 raft_consensus.cc:504] T 00000000000000000000000000000000 P ecdca47ce9f544afa6f7395b0ad3445e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecdca47ce9f544afa6f7395b0ad3445e" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 33353 } }
I20260806 22:50:18.921200 5117 leader_election.cc:302] T 00000000000000000000000000000000 P ecdca47ce9f544afa6f7395b0ad3445e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ecdca47ce9f544afa6f7395b0ad3445e; no voters:
I20260806 22:50:18.921397 5117 leader_election.cc:288] T 00000000000000000000000000000000 P ecdca47ce9f544afa6f7395b0ad3445e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:50:18.921425 5124 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ecdca47ce9f544afa6f7395b0ad3445e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:50:18.921546 5124 raft_consensus.cc:686] T 00000000000000000000000000000000 P ecdca47ce9f544afa6f7395b0ad3445e [term 1 LEADER]: Becoming Leader. State: Replica: ecdca47ce9f544afa6f7395b0ad3445e, State: Running, Role: LEADER
I20260806 22:50:18.921660 5117 sys_catalog.cc:564] T 00000000000000000000000000000000 P ecdca47ce9f544afa6f7395b0ad3445e [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:50:18.921627 5124 consensus_queue.cc:237] T 00000000000000000000000000000000 P ecdca47ce9f544afa6f7395b0ad3445e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecdca47ce9f544afa6f7395b0ad3445e" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 33353 } }
I20260806 22:50:18.921967 5126 sys_catalog.cc:455] T 00000000000000000000000000000000 P ecdca47ce9f544afa6f7395b0ad3445e [sys.catalog]: SysCatalogTable state changed. Reason: New leader ecdca47ce9f544afa6f7395b0ad3445e. Latest consensus state: current_term: 1 leader_uuid: "ecdca47ce9f544afa6f7395b0ad3445e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecdca47ce9f544afa6f7395b0ad3445e" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 33353 } } }
I20260806 22:50:18.921969 5125 sys_catalog.cc:455] T 00000000000000000000000000000000 P ecdca47ce9f544afa6f7395b0ad3445e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ecdca47ce9f544afa6f7395b0ad3445e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecdca47ce9f544afa6f7395b0ad3445e" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 33353 } } }
I20260806 22:50:18.922158 5125 sys_catalog.cc:458] T 00000000000000000000000000000000 P ecdca47ce9f544afa6f7395b0ad3445e [sys.catalog]: This master's current role is: LEADER
I20260806 22:50:18.922327 5126 sys_catalog.cc:458] T 00000000000000000000000000000000 P ecdca47ce9f544afa6f7395b0ad3445e [sys.catalog]: This master's current role is: LEADER
I20260806 22:50:18.922509 5131 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:50:18.923038 5131 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:50:18.924681 5131 catalog_manager.cc:1347] Generated new cluster ID: 3411a99a2f09436d80c2faa44ef66ddc
I20260806 22:50:18.924734 5131 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 22:50:18 dist-test-slave-1xrh krb5kdc[5015](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056618, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.28.218.254@KRBTEST.COM
I20260806 22:50:18.931458 5123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:18.919870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46130 (local address 127.28.218.254:33353)
0806 22:50:18.920042 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:18.920050 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:18.920074 (+ 24us) server_negotiation.cc:407] Connection header received
0806 22:50:18.920161 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:18.920165 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:18.920204 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:18.920347 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:18.921080 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:18.922386 (+ 1306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:18.923475 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:18.924075 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:18.924185 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:18.927181 (+ 2996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:50:18.927184 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:50:18.927185 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:50:18.927196 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:50:18.929715 (+ 2519us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:18.930129 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:18.930132 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:18.930132 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:18.930236 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:18.930449 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:18.930450 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:18.930450 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:18.930729 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:50:18.930871 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:18.930982 (+ 111us) server_negotiation.cc:299] Negotiation successful
0806 22:50:18.931238 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":91,"threads_started":1}
I20260806 22:50:18.943701 5131 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:50:18.944375 5131 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:50:18.949374 5131 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ecdca47ce9f544afa6f7395b0ad3445e: Generated new TSK 0
I20260806 22:50:18.949538 5131 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.28.218.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.28.218.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.28.218.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056586788339-29547-0/krb5kdc/oryx_127.28.218.193.keytab.
Entry for principal oryx/127.28.218.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056586788339-29547-0/krb5kdc/oryx_127.28.218.193.keytab.
I20260806 22:50:18.977782 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056586788339-29547-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056586788339-29547-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056586788339-29547-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056586788339-29547-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.218.193:0
--local_ip_for_outbound_sockets=127.28.218.193
--webserver_interface=127.28.218.193
--webserver_port=0
--tserver_master_addrs=127.28.218.254:33353
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056586788339-29547-0/krb5kdc/oryx_127.28.218.193.keytab
--principal=oryx/127.28.218.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.218.212:38031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:50:19.057329 5147 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:50:19.057525 5147 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:50:19.057548 5147 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:50:19.058671 5147 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:50:19.058712 5147 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:50:19.058732 5147 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 22:50:19.058785 5147 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.218.193
I20260806 22:50:19.060176 5147 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:38031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056586788339-29547-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056586788339-29547-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056586788339-29547-0/krb5kdc/oryx_127.28.218.193.keytab
--principal=oryx/127.28.218.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.218.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056586788339-29547-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.218.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.28.218.254:33353
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5147
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.218.193
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056586788339-29547-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:50:19.060395 5147 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:50:19.060602 5147 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:50:19.065282 5154 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:19.065371 5153 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:19.065300 5156 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:50:19.065634 5147 server_base.cc:1034] running on GCE node
I20260806 22:50:19.065883 5147 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:50:19.066114 5147 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:50:19.067255 5147 hybrid_clock.cc:648] HybridClock initialized: now 1786056619067232 us; error 36 us; skew 500 ppm
Aug 06 22:50:19 dist-test-slave-1xrh krb5kdc[5015](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056619, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.218.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:50:19.069612 5147 init.cc:377] Logged in from keytab as oryx/127.28.218.193@KRBTEST.COM (short username oryx)
I20260806 22:50:19.070078 5147 webserver.cc:460] Webserver started at http://127.28.218.193:38451/ using document root <none> and password file <none>
I20260806 22:50:19.070246 5147 fs_manager.cc:359] Metadata directory not provided
I20260806 22:50:19.070284 5147 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:50:19.070369 5147 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:50:19.071389 5147 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056586788339-29547-0/minicluster-data/ts-0/data/instance:
uuid: "589db803901b4d34aac64364c0164d6c"
format_stamp: "Formatted at 2026-08-06 22:50:19 on dist-test-slave-1xrh"
server_key: "3f648eda199c00fe9e8bab8999f2b256"
server_key_iv: "43ea8b4cd0a4ac081f80e869d38faa62"
server_key_version: "encryptionkey@0"
I20260806 22:50:19.071676 5147 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056586788339-29547-0/minicluster-data/ts-0/wal/instance:
uuid: "589db803901b4d34aac64364c0164d6c"
format_stamp: "Formatted at 2026-08-06 22:50:19 on dist-test-slave-1xrh"
server_key: "3f648eda199c00fe9e8bab8999f2b256"
server_key_iv: "43ea8b4cd0a4ac081f80e869d38faa62"
server_key_version: "encryptionkey@0"
I20260806 22:50:19.072840 5147 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:50:19.073436 5164 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:19.073609 5147 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:50:19.073689 5147 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056586788339-29547-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056586788339-29547-0/minicluster-data/ts-0/wal
uuid: "589db803901b4d34aac64364c0164d6c"
format_stamp: "Formatted at 2026-08-06 22:50:19 on dist-test-slave-1xrh"
server_key: "3f648eda199c00fe9e8bab8999f2b256"
server_key_iv: "43ea8b4cd0a4ac081f80e869d38faa62"
server_key_version: "encryptionkey@0"
I20260806 22:50:19.073740 5147 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056586788339-29547-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056586788339-29547-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056586788339-29547-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:50:19.095642 5147 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:50:19.097755 5147 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:50:19.097855 5147 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:50:19.098253 5147 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:50:19.098291 5147 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:19.098312 5147 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:50:19.098357 5147 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:19.104569 5147 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.193:42059
I20260806 22:50:19.104637 5289 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.193:42059 every 8 connection(s)
I20260806 22:50:19.104907 5147 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056586788339-29547-0/minicluster-data/ts-0/data/info.pb
Aug 06 22:50:19 dist-test-slave-1xrh krb5kdc[5015](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056619, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.218.193@KRBTEST.COM for oryx/127.28.218.254@KRBTEST.COM
I20260806 22:50:19.108201 5123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:19.098748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.193:51189 (local address 127.28.218.254:33353)
0806 22:50:19.098819 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:19.098821 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:19.098954 (+ 133us) server_negotiation.cc:407] Connection header received
0806 22:50:19.099407 (+ 453us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:19.099408 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:19.099424 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:19.099501 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:19.100301 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:19.101113 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:19.102567 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:19.102901 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:19.102981 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:19.105500 (+ 2519us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:50:19.105501 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:50:19.105501 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:50:19.105508 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:50:19.107042 (+ 1534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:19.107459 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:19.107460 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:19.107460 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:19.107510 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:19.107687 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:19.107688 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:19.107688 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:19.107785 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:50:19.107830 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:19.108076 (+ 246us) server_negotiation.cc:299] Negotiation successful
0806 22:50:19.108140 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 22:50:19.108322 5174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:19.098680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:33353 (local address 127.28.218.193:51189)
0806 22:50:19.098836 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:19.098869 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:19.099319 (+ 450us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:19.099590 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:19.099599 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:19.099941 (+ 342us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:19.100200 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:19.100206 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:19.101216 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:19.101217 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:19.102431 (+ 1214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:19.102434 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:19.102984 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:19.102985 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:19.103077 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:19.103636 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:50:19.103646 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:19.105408 (+ 1762us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:50:19.107110 (+ 1702us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:19.107115 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:19.107117 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:19.107395 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:19.107554 (+ 159us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:19.107554 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:19.107554 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:19.107634 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:19.107844 (+ 210us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:50:19.107849 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:50:19.107957 (+ 108us) client_negotiation.cc:769] Sending connection context
0806 22:50:19.108030 (+ 73us) client_negotiation.cc:240] Negotiation successful
0806 22:50:19.108137 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":68,"threads_started":1}
I20260806 22:50:19.108798 5290 heartbeater.cc:347] Connected to a master server at 127.28.218.254:33353
I20260806 22:50:19.108876 5290 heartbeater.cc:464] Registering TS with master...
I20260806 22:50:19.109048 5290 heartbeater.cc:511] Master 127.28.218.254:33353 requested a full tablet report, sending...
I20260806 22:50:19.109570 5061 ts_manager.cc:194] Registered new tserver with Master: 589db803901b4d34aac64364c0164d6c (127.28.218.193:42059)
I20260806 22:50:19.110450 5061 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.28.218.193@KRBTEST.COM'} at 127.28.218.193:51189
I20260806 22:50:19.112756 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 5147
I20260806 22:50:19.112895 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056586788339-29547-0/minicluster-data/ts-0/wal/instance
I20260806 22:50:19.113092 29547 external_mini_cluster.cc:1428] Setting key 154ea4f033b62ad4b4a181a3b3d8987c
I20260806 22:50:19.114910 5123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:19.109768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.193:37669 (local address 127.28.218.254:33353)
0806 22:50:19.109808 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:19.109809 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:19.109910 (+ 101us) server_negotiation.cc:407] Connection header received
0806 22:50:19.109977 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:19.109978 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:19.109985 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:19.110036 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:19.110336 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:19.110886 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:19.111640 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:19.111796 (+ 156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:19.111844 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:19.112509 (+ 665us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:50:19.112510 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:50:19.112510 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:50:19.112532 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:50:19.113839 (+ 1307us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:19.114160 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:19.114161 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:19.114161 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:19.114211 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:19.114434 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:19.114452 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:19.114452 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:19.114546 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:50:19.114615 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:19.114764 (+ 149us) server_negotiation.cc:299] Negotiation successful
0806 22:50:19.114831 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 22:50:19.114921 5296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:19.109731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:33353 (local address 127.28.218.193:37669)
0806 22:50:19.109853 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:19.109864 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:19.109924 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:19.110043 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:19.110043 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:19.110223 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:19.110276 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:19.110278 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:19.110981 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:19.110982 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:19.111578 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:19.111581 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:19.111855 (+ 274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:19.111856 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:19.111900 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:19.112188 (+ 288us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:50:19.112191 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:19.112425 (+ 234us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:50:19.113925 (+ 1500us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:19.113926 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:19.113926 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:19.114084 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:19.114275 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:19.114276 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:19.114276 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:19.114329 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:19.114609 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:50:19.114609 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:50:19.114652 (+ 43us) client_negotiation.cc:769] Sending connection context
0806 22:50:19.114730 (+ 78us) client_negotiation.cc:240] Negotiation successful
0806 22:50:19.114823 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":73,"threads_started":1}
WARNING: no policy specified for oryx/127.28.218.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.28.218.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.28.218.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056586788339-29547-0/krb5kdc/oryx_127.28.218.194.keytab.
Entry for principal oryx/127.28.218.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056586788339-29547-0/krb5kdc/oryx_127.28.218.194.keytab.
I20260806 22:50:19.138190 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056586788339-29547-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056586788339-29547-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056586788339-29547-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056586788339-29547-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.218.194:0
--local_ip_for_outbound_sockets=127.28.218.194
--webserver_interface=127.28.218.194
--webserver_port=0
--tserver_master_addrs=127.28.218.254:33353
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056586788339-29547-0/krb5kdc/oryx_127.28.218.194.keytab
--principal=oryx/127.28.218.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.218.212:38031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:50:19.218348 5299 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:50:19.218533 5299 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:50:19.218556 5299 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:50:19.219722 5299 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:50:19.219761 5299 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:50:19.219781 5299 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 22:50:19.219841 5299 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.218.194
I20260806 22:50:19.221256 5299 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:38031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056586788339-29547-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056586788339-29547-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056586788339-29547-0/krb5kdc/oryx_127.28.218.194.keytab
--principal=oryx/127.28.218.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.218.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056586788339-29547-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.218.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.28.218.254:33353
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5299
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.218.194
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056586788339-29547-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:50:19.221488 5299 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:50:19.221698 5299 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:50:19.226831 5305 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:19.226854 5307 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:19.226832 5304 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:50:19.226881 5299 server_base.cc:1034] running on GCE node
I20260806 22:50:19.227350 5299 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:50:19.227592 5299 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:50:19.228740 5299 hybrid_clock.cc:648] HybridClock initialized: now 1786056619228722 us; error 34 us; skew 500 ppm
Aug 06 22:50:19 dist-test-slave-1xrh krb5kdc[5015](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056619, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.218.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:50:19.231065 5299 init.cc:377] Logged in from keytab as oryx/127.28.218.194@KRBTEST.COM (short username oryx)
I20260806 22:50:19.231592 5299 webserver.cc:460] Webserver started at http://127.28.218.194:36167/ using document root <none> and password file <none>
I20260806 22:50:19.231841 5299 fs_manager.cc:359] Metadata directory not provided
I20260806 22:50:19.231889 5299 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:50:19.232059 5299 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:50:19.233099 5299 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056586788339-29547-0/minicluster-data/ts-1/data/instance:
uuid: "e2abad33dd7046b2bb589197ed70fa25"
format_stamp: "Formatted at 2026-08-06 22:50:19 on dist-test-slave-1xrh"
server_key: "b51b2658ba934152df5cc8b2afe41f23"
server_key_iv: "77b7be99480c7e1d0e59034f801b54c6"
server_key_version: "encryptionkey@0"
I20260806 22:50:19.233470 5299 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056586788339-29547-0/minicluster-data/ts-1/wal/instance:
uuid: "e2abad33dd7046b2bb589197ed70fa25"
format_stamp: "Formatted at 2026-08-06 22:50:19 on dist-test-slave-1xrh"
server_key: "b51b2658ba934152df5cc8b2afe41f23"
server_key_iv: "77b7be99480c7e1d0e59034f801b54c6"
server_key_version: "encryptionkey@0"
I20260806 22:50:19.234758 5299 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:50:19.235508 5314 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:19.235673 5299 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:19.235770 5299 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056586788339-29547-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056586788339-29547-0/minicluster-data/ts-1/wal
uuid: "e2abad33dd7046b2bb589197ed70fa25"
format_stamp: "Formatted at 2026-08-06 22:50:19 on dist-test-slave-1xrh"
server_key: "b51b2658ba934152df5cc8b2afe41f23"
server_key_iv: "77b7be99480c7e1d0e59034f801b54c6"
server_key_version: "encryptionkey@0"
I20260806 22:50:19.235845 5299 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056586788339-29547-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056586788339-29547-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056586788339-29547-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:50:19.244776 5299 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:50:19.246493 5299 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:50:19.246649 5299 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:50:19.247239 5299 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:50:19.247299 5299 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:19.247331 5299 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:50:19.247363 5299 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 22:50:19 dist-test-slave-1xrh krb5kdc[5015](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056619, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.218.194@KRBTEST.COM for oryx/127.28.218.254@KRBTEST.COM
I20260806 22:50:19.254758 5299 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.194:41871
I20260806 22:50:19.254787 5435 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.194:41871 every 8 connection(s)
I20260806 22:50:19.255156 5299 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056586788339-29547-0/minicluster-data/ts-1/data/info.pb
I20260806 22:50:19.256251 5123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:19.247596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.194:59027 (local address 127.28.218.254:33353)
0806 22:50:19.247686 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:19.247689 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:19.247902 (+ 213us) server_negotiation.cc:407] Connection header received
0806 22:50:19.248384 (+ 482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:19.248386 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:19.248400 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:19.248484 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:19.249012 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:19.249528 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:19.250502 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:19.250793 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:19.250875 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:19.253488 (+ 2613us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:50:19.253489 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:50:19.253490 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:50:19.253497 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:50:19.255072 (+ 1575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:19.255555 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:19.255556 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:19.255557 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:19.255611 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:19.255806 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:19.255807 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:19.255807 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:19.255898 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:50:19.255942 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:19.256119 (+ 177us) server_negotiation.cc:299] Negotiation successful
0806 22:50:19.256174 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 22:50:19.256402 5323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:19.247602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:33353 (local address 127.28.218.194:59027)
0806 22:50:19.247774 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:19.247811 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:19.248286 (+ 475us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:19.248529 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:19.248532 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:19.248746 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:19.248907 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:19.248911 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:19.249628 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:19.249629 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:19.250394 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:19.250397 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:19.250887 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:19.250887 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:19.251006 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:19.251529 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:50:19.251536 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:19.253400 (+ 1864us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:50:19.255174 (+ 1774us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:19.255179 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:19.255181 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:19.255423 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:19.255663 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:19.255663 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:19.255664 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:19.255748 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:19.255934 (+ 186us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:50:19.255937 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:50:19.256026 (+ 89us) client_negotiation.cc:769] Sending connection context
0806 22:50:19.256093 (+ 67us) client_negotiation.cc:240] Negotiation successful
0806 22:50:19.256220 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":106,"threads_started":1}
I20260806 22:50:19.256882 5436 heartbeater.cc:347] Connected to a master server at 127.28.218.254:33353
I20260806 22:50:19.256955 5436 heartbeater.cc:464] Registering TS with master...
I20260806 22:50:19.257136 5436 heartbeater.cc:511] Master 127.28.218.254:33353 requested a full tablet report, sending...
I20260806 22:50:19.257467 5063 ts_manager.cc:194] Registered new tserver with Master: e2abad33dd7046b2bb589197ed70fa25 (127.28.218.194:41871)
I20260806 22:50:19.257797 5063 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.28.218.194@KRBTEST.COM'} at 127.28.218.194:59027
I20260806 22:50:19.262710 5442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:19.257861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:33353 (local address 127.28.218.194:58367)
0806 22:50:19.257984 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:19.257994 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:19.258063 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:19.258191 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:19.258192 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:19.258392 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:19.258448 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:19.258450 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:19.259099 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:19.259099 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:19.259614 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:19.259615 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:19.259981 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:19.259981 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:19.260030 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:19.260421 (+ 391us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:50:19.260426 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:19.260697 (+ 271us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:50:19.261981 (+ 1284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:19.261982 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:19.261982 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:19.262130 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:19.262285 (+ 155us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:19.262285 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:19.262286 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:19.262329 (+ 43us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:19.262514 (+ 185us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:50:19.262514 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:50:19.262546 (+ 32us) client_negotiation.cc:769] Sending connection context
0806 22:50:19.262592 (+ 46us) client_negotiation.cc:240] Negotiation successful
0806 22:50:19.262653 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":75,"threads_started":1}
I20260806 22:50:19.262804 5123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:19.257911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.194:58367 (local address 127.28.218.254:33353)
0806 22:50:19.257948 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:19.257950 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:19.258046 (+ 96us) server_negotiation.cc:407] Connection header received
0806 22:50:19.258112 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:19.258113 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:19.258123 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:19.258182 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:19.258519 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:19.259012 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:19.259678 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:19.259904 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:19.259965 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:19.260784 (+ 819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:50:19.260785 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:50:19.260785 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:50:19.260790 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:50:19.261899 (+ 1109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:19.262190 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:19.262190 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:19.262190 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:19.262234 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:19.262363 (+ 129us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:19.262364 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:19.262364 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:19.262468 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:50:19.262509 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:19.262646 (+ 137us) server_negotiation.cc:299] Negotiation successful
0806 22:50:19.262717 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 22:50:19.262800 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 5299
I20260806 22:50:19.262900 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056586788339-29547-0/minicluster-data/ts-1/wal/instance
I20260806 22:50:19.263064 29547 external_mini_cluster.cc:1428] Setting key 9f310c7290b96b78f576e29885ce3509
WARNING: no policy specified for oryx/127.28.218.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.28.218.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.28.218.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056586788339-29547-0/krb5kdc/oryx_127.28.218.195.keytab.
Entry for principal oryx/127.28.218.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056586788339-29547-0/krb5kdc/oryx_127.28.218.195.keytab.
I20260806 22:50:19.287905 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056586788339-29547-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056586788339-29547-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056586788339-29547-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056586788339-29547-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.218.195:0
--local_ip_for_outbound_sockets=127.28.218.195
--webserver_interface=127.28.218.195
--webserver_port=0
--tserver_master_addrs=127.28.218.254:33353
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056586788339-29547-0/krb5kdc/oryx_127.28.218.195.keytab
--principal=oryx/127.28.218.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.218.212:38031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:50:19.370359 5445 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:50:19.370591 5445 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:50:19.370630 5445 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:50:19.371869 5445 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:50:19.371932 5445 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:50:19.371964 5445 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 22:50:19.372004 5445 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.218.195
I20260806 22:50:19.373505 5445 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:38031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056586788339-29547-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056586788339-29547-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056586788339-29547-0/krb5kdc/oryx_127.28.218.195.keytab
--principal=oryx/127.28.218.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.218.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056586788339-29547-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.218.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.28.218.254:33353
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5445
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.218.195
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056586788339-29547-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:50:19.373767 5445 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:50:19.373998 5445 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 22:50:19.378998 5445 server_base.cc:1034] running on GCE node
W20260806 22:50:19.378985 5451 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:19.378973 5450 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:19.378971 5453 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:50:19.379506 5445 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:50:19.379760 5445 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:50:19.380949 5445 hybrid_clock.cc:648] HybridClock initialized: now 1786056619380927 us; error 35 us; skew 500 ppm
Aug 06 22:50:19 dist-test-slave-1xrh krb5kdc[5015](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056619, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.218.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:50:19.383553 5445 init.cc:377] Logged in from keytab as oryx/127.28.218.195@KRBTEST.COM (short username oryx)
I20260806 22:50:19.384032 5445 webserver.cc:460] Webserver started at http://127.28.218.195:33025/ using document root <none> and password file <none>
I20260806 22:50:19.384270 5445 fs_manager.cc:359] Metadata directory not provided
I20260806 22:50:19.384341 5445 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:50:19.384491 5445 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:50:19.385509 5445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056586788339-29547-0/minicluster-data/ts-2/data/instance:
uuid: "3b7f0599c31b4cd798417dc934125fd0"
format_stamp: "Formatted at 2026-08-06 22:50:19 on dist-test-slave-1xrh"
server_key: "02cfbcd71a854bb1f2ebb11cee9122f0"
server_key_iv: "b661068cfd2b9deff302e59bf30cf972"
server_key_version: "encryptionkey@0"
I20260806 22:50:19.385851 5445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056586788339-29547-0/minicluster-data/ts-2/wal/instance:
uuid: "3b7f0599c31b4cd798417dc934125fd0"
format_stamp: "Formatted at 2026-08-06 22:50:19 on dist-test-slave-1xrh"
server_key: "02cfbcd71a854bb1f2ebb11cee9122f0"
server_key_iv: "b661068cfd2b9deff302e59bf30cf972"
server_key_version: "encryptionkey@0"
I20260806 22:50:19.387398 5445 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 22:50:19.388094 5460 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:19.388260 5445 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:50:19.388372 5445 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056586788339-29547-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056586788339-29547-0/minicluster-data/ts-2/wal
uuid: "3b7f0599c31b4cd798417dc934125fd0"
format_stamp: "Formatted at 2026-08-06 22:50:19 on dist-test-slave-1xrh"
server_key: "02cfbcd71a854bb1f2ebb11cee9122f0"
server_key_iv: "b661068cfd2b9deff302e59bf30cf972"
server_key_version: "encryptionkey@0"
I20260806 22:50:19.388443 5445 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056586788339-29547-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056586788339-29547-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056586788339-29547-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:50:19.400058 5445 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:50:19.402355 5445 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:50:19.402508 5445 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:50:19.402954 5445 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:50:19.403015 5445 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:19.403051 5445 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:50:19.403097 5445 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:19.409190 5445 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.195:39813
I20260806 22:50:19.409268 5579 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.195:39813 every 8 connection(s)
I20260806 22:50:19.409525 5445 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056586788339-29547-0/minicluster-data/ts-2/data/info.pb
Aug 06 22:50:19 dist-test-slave-1xrh krb5kdc[5015](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056619, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.218.195@KRBTEST.COM for oryx/127.28.218.254@KRBTEST.COM
I20260806 22:50:19.412181 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 5445
I20260806 22:50:19.412349 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056586788339-29547-0/minicluster-data/ts-2/wal/instance
I20260806 22:50:19.412600 29547 external_mini_cluster.cc:1428] Setting key 28e596fd30af619bd8c19b36c4bb08da
I20260806 22:50:19.412942 5123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:19.403371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.195:43553 (local address 127.28.218.254:33353)
0806 22:50:19.403412 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:19.403414 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:19.403660 (+ 246us) server_negotiation.cc:407] Connection header received
0806 22:50:19.404162 (+ 502us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:19.404163 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:19.404176 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:19.404234 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:19.405025 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:19.405801 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:19.407342 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:19.407536 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:19.407613 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:19.410126 (+ 2513us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:50:19.410127 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:50:19.410127 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:50:19.410133 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:50:19.411585 (+ 1452us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:19.411911 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:19.411911 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:19.411912 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:19.411958 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:19.412235 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:19.412236 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:19.412237 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:19.412406 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:50:19.412480 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:19.412762 (+ 282us) server_negotiation.cc:299] Negotiation successful
0806 22:50:19.412846 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 22:50:19.413079 5470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:19.403379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:33353 (local address 127.28.218.195:43553)
0806 22:50:19.403548 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:19.403579 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:19.404060 (+ 481us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:19.404294 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:19.404297 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:19.404672 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:19.404933 (+ 261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:19.404938 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:19.405906 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:19.405907 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:19.407224 (+ 1317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:19.407245 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:19.407618 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:19.407619 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:19.407753 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:19.408292 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:50:19.408302 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:19.410043 (+ 1741us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:50:19.411650 (+ 1607us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:19.411654 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:19.411656 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:19.411848 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:19.412075 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:19.412075 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:19.412076 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:19.412172 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:19.412482 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:50:19.412486 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:50:19.412600 (+ 114us) client_negotiation.cc:769] Sending connection context
0806 22:50:19.412716 (+ 116us) client_negotiation.cc:240] Negotiation successful
0806 22:50:19.412868 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":96,"threads_started":1}
I20260806 22:50:19.413568 5580 heartbeater.cc:347] Connected to a master server at 127.28.218.254:33353
I20260806 22:50:19.413648 5580 heartbeater.cc:464] Registering TS with master...
I20260806 22:50:19.413817 5580 heartbeater.cc:511] Master 127.28.218.254:33353 requested a full tablet report, sending...
I20260806 22:50:19.414170 5060 ts_manager.cc:194] Registered new tserver with Master: 3b7f0599c31b4cd798417dc934125fd0 (127.28.218.195:39813)
I20260806 22:50:19.414628 5060 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.28.218.195@KRBTEST.COM'} at 127.28.218.195:43553
I20260806 22:50:19.419536 5123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:19.414596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.195:56231 (local address 127.28.218.254:33353)
0806 22:50:19.414638 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:19.414640 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:19.414759 (+ 119us) server_negotiation.cc:407] Connection header received
0806 22:50:19.414839 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:19.414839 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:19.414847 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:19.414890 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:19.415224 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:19.415707 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:19.416312 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:19.416492 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:19.416541 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:19.417253 (+ 712us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:50:19.417254 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:50:19.417254 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:50:19.417260 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:50:19.418638 (+ 1378us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:19.418913 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:19.418914 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:19.418914 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:19.418957 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:19.419108 (+ 151us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:19.419109 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:19.419109 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:19.419235 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:50:19.419282 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:19.419394 (+ 112us) server_negotiation.cc:299] Negotiation successful
0806 22:50:19.419445 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 22:50:19.419549 5588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:19.414537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:33353 (local address 127.28.218.195:56231)
0806 22:50:19.414691 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:19.414699 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:19.414782 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:19.414902 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:19.414903 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:19.415073 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:19.415164 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:19.415165 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:19.415784 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:19.415785 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:19.416249 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:19.416251 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:19.416552 (+ 301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:19.416553 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:19.416596 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:19.416917 (+ 321us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:50:19.416921 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:19.417181 (+ 260us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:50:19.418715 (+ 1534us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:19.418716 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:19.418717 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:19.418866 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:19.419011 (+ 145us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:19.419012 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:19.419012 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:19.419072 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:19.419294 (+ 222us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:50:19.419294 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:50:19.419326 (+ 32us) client_negotiation.cc:769] Sending connection context
0806 22:50:19.419374 (+ 48us) client_negotiation.cc:240] Negotiation successful
0806 22:50:19.419449 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":78,"threads_started":1}
I20260806 22:50:19.424674 29547 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 22:50:19 dist-test-slave-1xrh krb5kdc[5015](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056619, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 22:50:19 dist-test-slave-1xrh krb5kdc[5015](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.28.218.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:50:19 dist-test-slave-1xrh krb5kdc[5015](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.28.218.254@KRBTEST.COM, Server not found in Kerberos database
W20260806 22:50:19.445721 5123 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:50:19.440669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40670 (local address 127.28.218.254:33353)
0806 22:50:19.440719 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:19.440721 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:19.440730 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:50:19.440796 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:19.440797 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:19.440806 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:19.440860 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:19.441352 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:19.441808 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:19.442520 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:19.442706 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:19.442778 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:19.445643 (+ 2865us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40670: BlockingRecv error: recv got EOF from 127.0.0.1:40670 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
Aug 06 22:50:19 dist-test-slave-1xrh krb5kdc[5015](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056619, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.28.218.254@KRBTEST.COM
I20260806 22:50:19.452868 5123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:19.446443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40676 (local address 127.28.218.254:33353)
0806 22:50:19.446528 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:19.446530 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:19.446616 (+ 86us) server_negotiation.cc:407] Connection header received
0806 22:50:19.446687 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:19.446688 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:19.446698 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:19.446763 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:19.447265 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:19.447697 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:19.448361 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:19.448539 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:19.448610 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:19.450533 (+ 1923us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:50:19.450534 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:50:19.450535 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:50:19.450541 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:50:19.451864 (+ 1323us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:19.452179 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:19.452180 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:19.452181 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:19.452230 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:19.452417 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:19.452417 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:19.452418 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:19.452516 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:50:19.452605 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:19.452708 (+ 103us) server_negotiation.cc:299] Negotiation successful
0806 22:50:19.452772 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 22:50:19.454324 5060 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40676:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:50:19.455817 5060 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:50:19.461813 5610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:19.458069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.194:41871 (local address 127.0.0.1:47640)
0806 22:50:19.458258 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:19.458270 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:19.458322 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:19.458568 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:19.458571 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:19.458572 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:50:19.458625 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:19.458626 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:19.459587 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:19.459588 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:19.460699 (+ 1111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:19.460702 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:19.461494 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:19.461495 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:19.461576 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:19.461590 (+ 14us) client_negotiation.cc:769] Sending connection context
0806 22:50:19.461685 (+ 95us) client_negotiation.cc:240] Negotiation successful
0806 22:50:19.461745 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":59,"threads_started":1}
I20260806 22:50:19.461867 5611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:19.458239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47640 (local address 127.28.218.194:41871)
0806 22:50:19.458398 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:19.458401 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:19.458408 (+ 7us) server_negotiation.cc:407] Connection header received
0806 22:50:19.458437 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:19.458438 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:19.458450 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:19.458555 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:50:19.458702 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:19.459504 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:19.460794 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:19.461386 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:19.461500 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:19.461568 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:19.461702 (+ 134us) server_negotiation.cc:299] Negotiation successful
0806 22:50:19.461750 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":102,"threads_started":1}
I20260806 22:50:19.461977 5607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:19.457905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.193:42059 (local address 127.0.0.1:44890)
0806 22:50:19.458196 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:19.458206 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:19.458246 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:19.458603 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:19.458603 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:19.458605 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:50:19.458662 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:19.458664 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:19.459999 (+ 1335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:19.460000 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:19.460815 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:19.460817 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:19.461743 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:19.461744 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:19.461801 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:19.461805 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:50:19.461834 (+ 29us) client_negotiation.cc:240] Negotiation successful
0806 22:50:19.461890 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":77,"threads_started":1}
I20260806 22:50:19.462390 5366 tablet_service.cc:1467] Processing CreateTablet for tablet 736b36c0ea05421d9bc7a95f84c5dbeb (DEFAULT_TABLE table=test-table [id=7e5bf214ab7243cd8f28eb992d028a3e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:50:19.462549 5608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:19.457974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33464 (local address 127.28.218.195:39813)
0806 22:50:19.458139 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:19.458144 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:19.458156 (+ 12us) server_negotiation.cc:407] Connection header received
0806 22:50:19.458445 (+ 289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:19.458448 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:19.458463 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:19.458542 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:50:19.458771 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:19.459851 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:19.461077 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:19.461789 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:19.461881 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:19.461930 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:19.462404 (+ 474us) server_negotiation.cc:299] Negotiation successful
0806 22:50:19.462452 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":73,"threads_started":1}
I20260806 22:50:19.462663 5609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:19.458017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44890 (local address 127.28.218.193:42059)
0806 22:50:19.458443 (+ 426us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:19.458447 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:19.458457 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:50:19.458495 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:19.458496 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:19.458510 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:19.458739 (+ 229us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:50:19.458800 (+ 61us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:19.459897 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:19.460913 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:19.461628 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:19.462327 (+ 699us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:19.462387 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:19.462410 (+ 23us) server_negotiation.cc:299] Negotiation successful
0806 22:50:19.462445 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":390,"thread_start_us":76,"threads_started":1}
I20260806 22:50:19.462720 5366 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 736b36c0ea05421d9bc7a95f84c5dbeb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:50:19.462754 5606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:19.457885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.195:39813 (local address 127.0.0.1:33464)
0806 22:50:19.458053 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:19.458093 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:19.458153 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:19.458666 (+ 513us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:19.458666 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:19.458667 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:50:19.458701 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:19.458702 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:19.459938 (+ 1236us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:19.459938 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:19.460996 (+ 1058us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:19.460998 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:19.461963 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:19.461964 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:19.462021 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:19.462025 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:50:19.462079 (+ 54us) client_negotiation.cc:240] Negotiation successful
0806 22:50:19.462672 (+ 593us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"mutex_wait_us":27,"thread_start_us":52,"threads_started":1}
I20260806 22:50:19.463397 5496 tablet_service.cc:1467] Processing CreateTablet for tablet 736b36c0ea05421d9bc7a95f84c5dbeb (DEFAULT_TABLE table=test-table [id=7e5bf214ab7243cd8f28eb992d028a3e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:50:19.463743 5496 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 736b36c0ea05421d9bc7a95f84c5dbeb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:50:19.466774 5612 tablet_bootstrap.cc:492] T 736b36c0ea05421d9bc7a95f84c5dbeb P e2abad33dd7046b2bb589197ed70fa25: Bootstrap starting.
I20260806 22:50:19.467402 5612 tablet_bootstrap.cc:654] T 736b36c0ea05421d9bc7a95f84c5dbeb P e2abad33dd7046b2bb589197ed70fa25: Neither blocks nor log segments found. Creating new log.
I20260806 22:50:19.467410 5613 tablet_bootstrap.cc:492] T 736b36c0ea05421d9bc7a95f84c5dbeb P 3b7f0599c31b4cd798417dc934125fd0: Bootstrap starting.
I20260806 22:50:19.467504 5217 tablet_service.cc:1467] Processing CreateTablet for tablet 736b36c0ea05421d9bc7a95f84c5dbeb (DEFAULT_TABLE table=test-table [id=7e5bf214ab7243cd8f28eb992d028a3e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:50:19.467769 5612 log.cc:824] T 736b36c0ea05421d9bc7a95f84c5dbeb P e2abad33dd7046b2bb589197ed70fa25: Log is configured to *not* fsync() on all Append() calls
I20260806 22:50:19.467885 5217 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 736b36c0ea05421d9bc7a95f84c5dbeb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:50:19.468113 5613 tablet_bootstrap.cc:654] T 736b36c0ea05421d9bc7a95f84c5dbeb P 3b7f0599c31b4cd798417dc934125fd0: Neither blocks nor log segments found. Creating new log.
I20260806 22:50:19.468406 5613 log.cc:824] T 736b36c0ea05421d9bc7a95f84c5dbeb P 3b7f0599c31b4cd798417dc934125fd0: Log is configured to *not* fsync() on all Append() calls
I20260806 22:50:19.468472 5612 tablet_bootstrap.cc:492] T 736b36c0ea05421d9bc7a95f84c5dbeb P e2abad33dd7046b2bb589197ed70fa25: No bootstrap required, opened a new log
I20260806 22:50:19.468538 5612 ts_tablet_manager.cc:1397] T 736b36c0ea05421d9bc7a95f84c5dbeb P e2abad33dd7046b2bb589197ed70fa25: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:50:19.469138 5613 tablet_bootstrap.cc:492] T 736b36c0ea05421d9bc7a95f84c5dbeb P 3b7f0599c31b4cd798417dc934125fd0: No bootstrap required, opened a new log
I20260806 22:50:19.469201 5613 ts_tablet_manager.cc:1397] T 736b36c0ea05421d9bc7a95f84c5dbeb P 3b7f0599c31b4cd798417dc934125fd0: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:50:19.470638 5612 raft_consensus.cc:348] T 736b36c0ea05421d9bc7a95f84c5dbeb P e2abad33dd7046b2bb589197ed70fa25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b7f0599c31b4cd798417dc934125fd0" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 39813 } } peers { permanent_uuid: "589db803901b4d34aac64364c0164d6c" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 42059 } } peers { permanent_uuid: "e2abad33dd7046b2bb589197ed70fa25" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 41871 } }
I20260806 22:50:19.470664 5613 raft_consensus.cc:348] T 736b36c0ea05421d9bc7a95f84c5dbeb P 3b7f0599c31b4cd798417dc934125fd0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b7f0599c31b4cd798417dc934125fd0" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 39813 } } peers { permanent_uuid: "589db803901b4d34aac64364c0164d6c" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 42059 } } peers { permanent_uuid: "e2abad33dd7046b2bb589197ed70fa25" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 41871 } }
I20260806 22:50:19.470786 5613 raft_consensus.cc:374] T 736b36c0ea05421d9bc7a95f84c5dbeb P 3b7f0599c31b4cd798417dc934125fd0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:50:19.470777 5612 raft_consensus.cc:374] T 736b36c0ea05421d9bc7a95f84c5dbeb P e2abad33dd7046b2bb589197ed70fa25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:50:19.470808 5613 raft_consensus.cc:729] T 736b36c0ea05421d9bc7a95f84c5dbeb P 3b7f0599c31b4cd798417dc934125fd0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b7f0599c31b4cd798417dc934125fd0, State: Initialized, Role: FOLLOWER
I20260806 22:50:19.470809 5612 raft_consensus.cc:729] T 736b36c0ea05421d9bc7a95f84c5dbeb P e2abad33dd7046b2bb589197ed70fa25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2abad33dd7046b2bb589197ed70fa25, State: Initialized, Role: FOLLOWER
I20260806 22:50:19.470907 5613 consensus_queue.cc:260] T 736b36c0ea05421d9bc7a95f84c5dbeb P 3b7f0599c31b4cd798417dc934125fd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b7f0599c31b4cd798417dc934125fd0" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 39813 } } peers { permanent_uuid: "589db803901b4d34aac64364c0164d6c" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 42059 } } peers { permanent_uuid: "e2abad33dd7046b2bb589197ed70fa25" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 41871 } }
I20260806 22:50:19.470932 5612 consensus_queue.cc:260] T 736b36c0ea05421d9bc7a95f84c5dbeb P e2abad33dd7046b2bb589197ed70fa25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b7f0599c31b4cd798417dc934125fd0" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 39813 } } peers { permanent_uuid: "589db803901b4d34aac64364c0164d6c" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 42059 } } peers { permanent_uuid: "e2abad33dd7046b2bb589197ed70fa25" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 41871 } }
I20260806 22:50:19.471170 5613 ts_tablet_manager.cc:1428] T 736b36c0ea05421d9bc7a95f84c5dbeb P 3b7f0599c31b4cd798417dc934125fd0: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:50:19.471220 5580 heartbeater.cc:503] Master 127.28.218.254:33353 was elected leader, sending a full tablet report...
I20260806 22:50:19.471225 5612 ts_tablet_manager.cc:1428] T 736b36c0ea05421d9bc7a95f84c5dbeb P e2abad33dd7046b2bb589197ed70fa25: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 22:50:19.471323 5436 heartbeater.cc:503] Master 127.28.218.254:33353 was elected leader, sending a full tablet report...
I20260806 22:50:19.471841 5618 tablet_bootstrap.cc:492] T 736b36c0ea05421d9bc7a95f84c5dbeb P 589db803901b4d34aac64364c0164d6c: Bootstrap starting.
I20260806 22:50:19.472476 5618 tablet_bootstrap.cc:654] T 736b36c0ea05421d9bc7a95f84c5dbeb P 589db803901b4d34aac64364c0164d6c: Neither blocks nor log segments found. Creating new log.
I20260806 22:50:19.472746 5618 log.cc:824] T 736b36c0ea05421d9bc7a95f84c5dbeb P 589db803901b4d34aac64364c0164d6c: Log is configured to *not* fsync() on all Append() calls
I20260806 22:50:19.473376 5618 tablet_bootstrap.cc:492] T 736b36c0ea05421d9bc7a95f84c5dbeb P 589db803901b4d34aac64364c0164d6c: No bootstrap required, opened a new log
I20260806 22:50:19.473445 5618 ts_tablet_manager.cc:1397] T 736b36c0ea05421d9bc7a95f84c5dbeb P 589db803901b4d34aac64364c0164d6c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:50:19.475114 5618 raft_consensus.cc:348] T 736b36c0ea05421d9bc7a95f84c5dbeb P 589db803901b4d34aac64364c0164d6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b7f0599c31b4cd798417dc934125fd0" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 39813 } } peers { permanent_uuid: "589db803901b4d34aac64364c0164d6c" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 42059 } } peers { permanent_uuid: "e2abad33dd7046b2bb589197ed70fa25" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 41871 } }
I20260806 22:50:19.475286 5618 raft_consensus.cc:374] T 736b36c0ea05421d9bc7a95f84c5dbeb P 589db803901b4d34aac64364c0164d6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:50:19.475307 5618 raft_consensus.cc:729] T 736b36c0ea05421d9bc7a95f84c5dbeb P 589db803901b4d34aac64364c0164d6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 589db803901b4d34aac64364c0164d6c, State: Initialized, Role: FOLLOWER
I20260806 22:50:19.475422 5618 consensus_queue.cc:260] T 736b36c0ea05421d9bc7a95f84c5dbeb P 589db803901b4d34aac64364c0164d6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b7f0599c31b4cd798417dc934125fd0" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 39813 } } peers { permanent_uuid: "589db803901b4d34aac64364c0164d6c" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 42059 } } peers { permanent_uuid: "e2abad33dd7046b2bb589197ed70fa25" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 41871 } }
I20260806 22:50:19.475643 5618 ts_tablet_manager.cc:1428] T 736b36c0ea05421d9bc7a95f84c5dbeb P 589db803901b4d34aac64364c0164d6c: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260806 22:50:19.475718 5290 heartbeater.cc:503] Master 127.28.218.254:33353 was elected leader, sending a full tablet report...
I20260806 22:50:19.498493 5616 raft_consensus.cc:482] T 736b36c0ea05421d9bc7a95f84c5dbeb P 3b7f0599c31b4cd798417dc934125fd0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:50:19.498589 5616 raft_consensus.cc:504] T 736b36c0ea05421d9bc7a95f84c5dbeb P 3b7f0599c31b4cd798417dc934125fd0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b7f0599c31b4cd798417dc934125fd0" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 39813 } } peers { permanent_uuid: "589db803901b4d34aac64364c0164d6c" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 42059 } } peers { permanent_uuid: "e2abad33dd7046b2bb589197ed70fa25" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 41871 } }
I20260806 22:50:19.498818 5616 leader_election.cc:288] T 736b36c0ea05421d9bc7a95f84c5dbeb P 3b7f0599c31b4cd798417dc934125fd0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 589db803901b4d34aac64364c0164d6c (127.28.218.193:42059), e2abad33dd7046b2bb589197ed70fa25 (127.28.218.194:41871)
I20260806 22:50:19.502377 5611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:19.499138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.195:58213 (local address 127.28.218.194:41871)
0806 22:50:19.499179 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:19.499180 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:19.499188 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:50:19.499214 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:19.499214 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:19.499221 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:19.499271 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:50:19.499469 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:19.500166 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:19.501478 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:19.502045 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:19.502116 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:19.502124 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:19.502282 (+ 158us) server_negotiation.cc:299] Negotiation successful
0806 22:50:19.502316 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 22:50:19.502377 5609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:19.499075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.195:49965 (local address 127.28.218.193:42059)
0806 22:50:19.499156 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:19.499158 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:19.499443 (+ 285us) server_negotiation.cc:407] Connection header received
0806 22:50:19.499497 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:19.499497 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:19.499507 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:19.499567 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:50:19.499696 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:19.500319 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:19.501533 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:19.502042 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:19.502116 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:19.502124 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:19.502282 (+ 158us) server_negotiation.cc:299] Negotiation successful
0806 22:50:19.502315 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260806 22:50:19.502424 5470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:19.499003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.194:41871 (local address 127.28.218.195:58213)
0806 22:50:19.499041 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:19.499052 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:19.499089 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:19.499296 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:19.499297 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:19.499300 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:50:19.499385 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:19.499387 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:19.500244 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:19.500245 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:19.501378 (+ 1133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:19.501380 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:19.502127 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:19.502128 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:19.502209 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:19.502212 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:50:19.502268 (+ 56us) client_negotiation.cc:240] Negotiation successful
0806 22:50:19.502310 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22,"mutex_wait_us":26}
I20260806 22:50:19.502434 5621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:19.499003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.193:42059 (local address 127.28.218.195:49965)
0806 22:50:19.499347 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:19.499358 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:19.499434 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:19.499579 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:19.499579 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:19.499580 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:50:19.499637 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:19.499638 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:19.500383 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:19.500384 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:19.501442 (+ 1058us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:19.501462 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:19.502127 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:19.502128 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:19.502209 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:19.502213 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:50:19.502268 (+ 55us) client_negotiation.cc:240] Negotiation successful
0806 22:50:19.502309 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":330,"mutex_wait_us":167,"thread_start_us":96,"threads_started":1}
I20260806 22:50:19.502815 5389 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "736b36c0ea05421d9bc7a95f84c5dbeb" candidate_uuid: "3b7f0599c31b4cd798417dc934125fd0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2abad33dd7046b2bb589197ed70fa25" is_pre_election: true
I20260806 22:50:19.502976 5389 raft_consensus.cc:2454] T 736b36c0ea05421d9bc7a95f84c5dbeb P e2abad33dd7046b2bb589197ed70fa25 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b7f0599c31b4cd798417dc934125fd0 in term 0.
I20260806 22:50:19.503074 5242 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "736b36c0ea05421d9bc7a95f84c5dbeb" candidate_uuid: "3b7f0599c31b4cd798417dc934125fd0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "589db803901b4d34aac64364c0164d6c" is_pre_election: true
I20260806 22:50:19.503193 5464 leader_election.cc:302] T 736b36c0ea05421d9bc7a95f84c5dbeb P 3b7f0599c31b4cd798417dc934125fd0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b7f0599c31b4cd798417dc934125fd0, e2abad33dd7046b2bb589197ed70fa25; no voters:
I20260806 22:50:19.503243 5242 raft_consensus.cc:2454] T 736b36c0ea05421d9bc7a95f84c5dbeb P 589db803901b4d34aac64364c0164d6c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b7f0599c31b4cd798417dc934125fd0 in term 0.
I20260806 22:50:19.503304 5616 raft_consensus.cc:2781] T 736b36c0ea05421d9bc7a95f84c5dbeb P 3b7f0599c31b4cd798417dc934125fd0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:50:19.503343 5616 raft_consensus.cc:482] T 736b36c0ea05421d9bc7a95f84c5dbeb P 3b7f0599c31b4cd798417dc934125fd0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:50:19.503362 5616 raft_consensus.cc:3037] T 736b36c0ea05421d9bc7a95f84c5dbeb P 3b7f0599c31b4cd798417dc934125fd0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:50:19.504204 5616 raft_consensus.cc:504] T 736b36c0ea05421d9bc7a95f84c5dbeb P 3b7f0599c31b4cd798417dc934125fd0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b7f0599c31b4cd798417dc934125fd0" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 39813 } } peers { permanent_uuid: "589db803901b4d34aac64364c0164d6c" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 42059 } } peers { permanent_uuid: "e2abad33dd7046b2bb589197ed70fa25" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 41871 } }
I20260806 22:50:19.504326 5616 leader_election.cc:288] T 736b36c0ea05421d9bc7a95f84c5dbeb P 3b7f0599c31b4cd798417dc934125fd0 [CANDIDATE]: Term 1 election: Requested vote from peers 589db803901b4d34aac64364c0164d6c (127.28.218.193:42059), e2abad33dd7046b2bb589197ed70fa25 (127.28.218.194:41871)
I20260806 22:50:19.504511 5389 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "736b36c0ea05421d9bc7a95f84c5dbeb" candidate_uuid: "3b7f0599c31b4cd798417dc934125fd0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2abad33dd7046b2bb589197ed70fa25"
I20260806 22:50:19.504511 5242 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "736b36c0ea05421d9bc7a95f84c5dbeb" candidate_uuid: "3b7f0599c31b4cd798417dc934125fd0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "589db803901b4d34aac64364c0164d6c"
I20260806 22:50:19.504586 5242 raft_consensus.cc:3037] T 736b36c0ea05421d9bc7a95f84c5dbeb P 589db803901b4d34aac64364c0164d6c [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:50:19.504586 5389 raft_consensus.cc:3037] T 736b36c0ea05421d9bc7a95f84c5dbeb P e2abad33dd7046b2bb589197ed70fa25 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:50:19.505373 5389 raft_consensus.cc:2454] T 736b36c0ea05421d9bc7a95f84c5dbeb P e2abad33dd7046b2bb589197ed70fa25 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b7f0599c31b4cd798417dc934125fd0 in term 1.
I20260806 22:50:19.505373 5242 raft_consensus.cc:2454] T 736b36c0ea05421d9bc7a95f84c5dbeb P 589db803901b4d34aac64364c0164d6c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b7f0599c31b4cd798417dc934125fd0 in term 1.
I20260806 22:50:19.505537 5464 leader_election.cc:302] T 736b36c0ea05421d9bc7a95f84c5dbeb P 3b7f0599c31b4cd798417dc934125fd0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b7f0599c31b4cd798417dc934125fd0, e2abad33dd7046b2bb589197ed70fa25; no voters:
I20260806 22:50:19.505625 5616 raft_consensus.cc:2781] T 736b36c0ea05421d9bc7a95f84c5dbeb P 3b7f0599c31b4cd798417dc934125fd0 [term 1 FOLLOWER]: Leader election won for term 1
W20260806 22:50:19.505640 5437 tablet.cc:2369] T 736b36c0ea05421d9bc7a95f84c5dbeb P e2abad33dd7046b2bb589197ed70fa25: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:50:19.505762 5616 raft_consensus.cc:686] T 736b36c0ea05421d9bc7a95f84c5dbeb P 3b7f0599c31b4cd798417dc934125fd0 [term 1 LEADER]: Becoming Leader. State: Replica: 3b7f0599c31b4cd798417dc934125fd0, State: Running, Role: LEADER
I20260806 22:50:19.505848 5616 consensus_queue.cc:237] T 736b36c0ea05421d9bc7a95f84c5dbeb P 3b7f0599c31b4cd798417dc934125fd0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b7f0599c31b4cd798417dc934125fd0" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 39813 } } peers { permanent_uuid: "589db803901b4d34aac64364c0164d6c" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 42059 } } peers { permanent_uuid: "e2abad33dd7046b2bb589197ed70fa25" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 41871 } }
I20260806 22:50:19.506574 5060 catalog_manager.cc:5495] T 736b36c0ea05421d9bc7a95f84c5dbeb P 3b7f0599c31b4cd798417dc934125fd0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3b7f0599c31b4cd798417dc934125fd0 (127.28.218.195). New cstate: current_term: 1 leader_uuid: "3b7f0599c31b4cd798417dc934125fd0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b7f0599c31b4cd798417dc934125fd0" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 39813 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "589db803901b4d34aac64364c0164d6c" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 42059 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2abad33dd7046b2bb589197ed70fa25" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 41871 } health_report { overall_health: UNKNOWN } } }
I20260806 22:50:19.518440 5608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:19.516090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33470 (local address 127.28.218.195:39813)
0806 22:50:19.516140 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:19.516142 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:19.516151 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:50:19.516211 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:19.516212 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:19.516227 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:19.516322 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:50:19.516459 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:19.516896 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:19.517625 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:19.517882 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:19.517944 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:19.518125 (+ 181us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:50:19.518220 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:50:19.518315 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:19.518351 (+ 36us) server_negotiation.cc:299] Negotiation successful
0806 22:50:19.518369 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260806 22:50:19.519927 5389 raft_consensus.cc:1261] T 736b36c0ea05421d9bc7a95f84c5dbeb P e2abad33dd7046b2bb589197ed70fa25 [term 1 FOLLOWER]: Refusing update from remote peer 3b7f0599c31b4cd798417dc934125fd0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:50:19.519982 5242 raft_consensus.cc:1261] T 736b36c0ea05421d9bc7a95f84c5dbeb P 589db803901b4d34aac64364c0164d6c [term 1 FOLLOWER]: Refusing update from remote peer 3b7f0599c31b4cd798417dc934125fd0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:50:19.520236 5616 consensus_queue.cc:1035] T 736b36c0ea05421d9bc7a95f84c5dbeb P 3b7f0599c31b4cd798417dc934125fd0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2abad33dd7046b2bb589197ed70fa25" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 41871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:50:19.520367 5622 consensus_queue.cc:1035] T 736b36c0ea05421d9bc7a95f84c5dbeb P 3b7f0599c31b4cd798417dc934125fd0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "589db803901b4d34aac64364c0164d6c" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 42059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:50:19.522665 5626 mvcc.cc:205] Tried to move back new op lower bound from 7315687913551458304 to 7315687913496547328. Current Snapshot: MvccSnapshot[applied={T|T < 7315687913551458304}]
I20260806 22:50:19.523190 5625 mvcc.cc:205] Tried to move back new op lower bound from 7315687913551458304 to 7315687913496547328. Current Snapshot: MvccSnapshot[applied={T|T < 7315687913551458304}]
I20260806 22:50:19.525126 5061 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40676:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:50:19.525197 5061 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40676:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:50:19.525876 5061 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ecdca47ce9f544afa6f7395b0ad3445e: Sending DeleteTablet for 3 replicas of tablet 736b36c0ea05421d9bc7a95f84c5dbeb
I20260806 22:50:19.526165 5496 tablet_service.cc:1514] Processing DeleteTablet for tablet 736b36c0ea05421d9bc7a95f84c5dbeb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:50:19 UTC) from {username='oryx', principal='oryx/127.28.218.254@KRBTEST.COM'} at 127.0.0.1:33464
I20260806 22:50:19.526191 5366 tablet_service.cc:1514] Processing DeleteTablet for tablet 736b36c0ea05421d9bc7a95f84c5dbeb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:50:19 UTC) from {username='oryx', principal='oryx/127.28.218.254@KRBTEST.COM'} at 127.0.0.1:47640
I20260806 22:50:19.526196 5217 tablet_service.cc:1514] Processing DeleteTablet for tablet 736b36c0ea05421d9bc7a95f84c5dbeb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:50:19 UTC) from {username='oryx', principal='oryx/127.28.218.254@KRBTEST.COM'} at 127.0.0.1:44890
I20260806 22:50:19.526376 5636 tablet_replica.cc:330] T 736b36c0ea05421d9bc7a95f84c5dbeb P 3b7f0599c31b4cd798417dc934125fd0: stopping tablet replica
I20260806 22:50:19.526489 5636 raft_consensus.cc:2229] T 736b36c0ea05421d9bc7a95f84c5dbeb P 3b7f0599c31b4cd798417dc934125fd0 [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:50:19.526604 5636 raft_consensus.cc:2258] T 736b36c0ea05421d9bc7a95f84c5dbeb P 3b7f0599c31b4cd798417dc934125fd0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:50:19.526696 5635 tablet_replica.cc:330] T 736b36c0ea05421d9bc7a95f84c5dbeb P e2abad33dd7046b2bb589197ed70fa25: stopping tablet replica
I20260806 22:50:19.526671 5061 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40676:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 22:50:19.526775 5635 raft_consensus.cc:2229] T 736b36c0ea05421d9bc7a95f84c5dbeb P e2abad33dd7046b2bb589197ed70fa25 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260806 22:50:19.526796 5061 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:50:19.527088 5636 ts_tablet_manager.cc:1905] T 736b36c0ea05421d9bc7a95f84c5dbeb P 3b7f0599c31b4cd798417dc934125fd0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:50:19.528173 5496 tablet_service.cc:1467] Processing CreateTablet for tablet 8b7b7bdfb66e438f9225f9f5ed67f6ca (DEFAULT_TABLE table=test-table [id=00f76a94f6fa4bcaae86038662d11925]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:50:19.528170 5366 tablet_service.cc:1467] Processing CreateTablet for tablet 8b7b7bdfb66e438f9225f9f5ed67f6ca (DEFAULT_TABLE table=test-table [id=00f76a94f6fa4bcaae86038662d11925]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:50:19.528277 5366 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b7b7bdfb66e438f9225f9f5ed67f6ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:50:19.528277 5496 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b7b7bdfb66e438f9225f9f5ed67f6ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:50:19.529526 5613 tablet_bootstrap.cc:492] T 8b7b7bdfb66e438f9225f9f5ed67f6ca P 3b7f0599c31b4cd798417dc934125fd0: Bootstrap starting.
I20260806 22:50:19.529547 5612 tablet_bootstrap.cc:492] T 8b7b7bdfb66e438f9225f9f5ed67f6ca P e2abad33dd7046b2bb589197ed70fa25: Bootstrap starting.
I20260806 22:50:19.529731 5217 tablet_service.cc:1467] Processing CreateTablet for tablet 8b7b7bdfb66e438f9225f9f5ed67f6ca (DEFAULT_TABLE table=test-table [id=00f76a94f6fa4bcaae86038662d11925]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:50:19.529827 5217 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b7b7bdfb66e438f9225f9f5ed67f6ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:50:19.529979 5612 tablet_bootstrap.cc:654] T 8b7b7bdfb66e438f9225f9f5ed67f6ca P e2abad33dd7046b2bb589197ed70fa25: Neither blocks nor log segments found. Creating new log.
I20260806 22:50:19.529979 5613 tablet_bootstrap.cc:654] T 8b7b7bdfb66e438f9225f9f5ed67f6ca P 3b7f0599c31b4cd798417dc934125fd0: Neither blocks nor log segments found. Creating new log.
I20260806 22:50:19.530287 5636 ts_tablet_manager.cc:1918] T 736b36c0ea05421d9bc7a95f84c5dbeb P 3b7f0599c31b4cd798417dc934125fd0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:50:19.530359 5636 log.cc:1196] T 736b36c0ea05421d9bc7a95f84c5dbeb P 3b7f0599c31b4cd798417dc934125fd0: Deleting WAL directory at /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056586788339-29547-0/minicluster-data/ts-2/wal/wals/736b36c0ea05421d9bc7a95f84c5dbeb
I20260806 22:50:19.530660 5636 ts_tablet_manager.cc:1939] T 736b36c0ea05421d9bc7a95f84c5dbeb P 3b7f0599c31b4cd798417dc934125fd0: Deleting consensus metadata
I20260806 22:50:19.531095 5047 catalog_manager.cc:4841] TS 3b7f0599c31b4cd798417dc934125fd0 (127.28.218.195:39813): tablet 736b36c0ea05421d9bc7a95f84c5dbeb (table test-table [id=7e5bf214ab7243cd8f28eb992d028a3e]) successfully deleted
I20260806 22:50:19.531491 5613 tablet_bootstrap.cc:492] T 8b7b7bdfb66e438f9225f9f5ed67f6ca P 3b7f0599c31b4cd798417dc934125fd0: No bootstrap required, opened a new log
I20260806 22:50:19.531523 5618 tablet_bootstrap.cc:492] T 8b7b7bdfb66e438f9225f9f5ed67f6ca P 589db803901b4d34aac64364c0164d6c: Bootstrap starting.
I20260806 22:50:19.531534 5612 tablet_bootstrap.cc:492] T 8b7b7bdfb66e438f9225f9f5ed67f6ca P e2abad33dd7046b2bb589197ed70fa25: No bootstrap required, opened a new log
I20260806 22:50:19.531544 5613 ts_tablet_manager.cc:1397] T 8b7b7bdfb66e438f9225f9f5ed67f6ca P 3b7f0599c31b4cd798417dc934125fd0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:50:19.531572 5612 ts_tablet_manager.cc:1397] T 8b7b7bdfb66e438f9225f9f5ed67f6ca P e2abad33dd7046b2bb589197ed70fa25: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:50:19.531697 5613 raft_consensus.cc:348] T 8b7b7bdfb66e438f9225f9f5ed67f6ca P 3b7f0599c31b4cd798417dc934125fd0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "589db803901b4d34aac64364c0164d6c" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 42059 } } peers { permanent_uuid: "e2abad33dd7046b2bb589197ed70fa25" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 41871 } } peers { permanent_uuid: "3b7f0599c31b4cd798417dc934125fd0" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 39813 } }
I20260806 22:50:19.531775 5613 raft_consensus.cc:374] T 8b7b7bdfb66e438f9225f9f5ed67f6ca P 3b7f0599c31b4cd798417dc934125fd0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:50:19.531800 5613 raft_consensus.cc:729] T 8b7b7bdfb66e438f9225f9f5ed67f6ca P 3b7f0599c31b4cd798417dc934125fd0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b7f0599c31b4cd798417dc934125fd0, State: Initialized, Role: FOLLOWER
I20260806 22:50:19.531852 5613 consensus_queue.cc:260] T 8b7b7bdfb66e438f9225f9f5ed67f6ca P 3b7f0599c31b4cd798417dc934125fd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "589db803901b4d34aac64364c0164d6c" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 42059 } } peers { permanent_uuid: "e2abad33dd7046b2bb589197ed70fa25" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 41871 } } peers { permanent_uuid: "3b7f0599c31b4cd798417dc934125fd0" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 39813 } }
I20260806 22:50:19.531944 5618 tablet_bootstrap.cc:654] T 8b7b7bdfb66e438f9225f9f5ed67f6ca P 589db803901b4d34aac64364c0164d6c: Neither blocks nor log segments found. Creating new log.
I20260806 22:50:19.531961 5613 ts_tablet_manager.cc:1428] T 8b7b7bdfb66e438f9225f9f5ed67f6ca P 3b7f0599c31b4cd798417dc934125fd0: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:19.531977 5635 raft_consensus.cc:2258] T 736b36c0ea05421d9bc7a95f84c5dbeb P e2abad33dd7046b2bb589197ed70fa25 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:50:19.531961 5612 raft_consensus.cc:348] T 8b7b7bdfb66e438f9225f9f5ed67f6ca P e2abad33dd7046b2bb589197ed70fa25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "589db803901b4d34aac64364c0164d6c" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 42059 } } peers { permanent_uuid: "e2abad33dd7046b2bb589197ed70fa25" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 41871 } } peers { permanent_uuid: "3b7f0599c31b4cd798417dc934125fd0" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 39813 } }
I20260806 22:50:19.532032 5612 raft_consensus.cc:374] T 8b7b7bdfb66e438f9225f9f5ed67f6ca P e2abad33dd7046b2bb589197ed70fa25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:50:19.532059 5612 raft_consensus.cc:729] T 8b7b7bdfb66e438f9225f9f5ed67f6ca P e2abad33dd7046b2bb589197ed70fa25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2abad33dd7046b2bb589197ed70fa25, State: Initialized, Role: FOLLOWER
I20260806 22:50:19.532109 5612 consensus_queue.cc:260] T 8b7b7bdfb66e438f9225f9f5ed67f6ca P e2abad33dd7046b2bb589197ed70fa25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "589db803901b4d34aac64364c0164d6c" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 42059 } } peers { permanent_uuid: "e2abad33dd7046b2bb589197ed70fa25" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 41871 } } peers { permanent_uuid: "3b7f0599c31b4cd798417dc934125fd0" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 39813 } }
I20260806 22:50:19.532135 5637 tablet_replica.cc:330] T 736b36c0ea05421d9bc7a95f84c5dbeb P 589db803901b4d34aac64364c0164d6c: stopping tablet replica
I20260806 22:50:19.532207 5612 ts_tablet_manager.cc:1428] T 8b7b7bdfb66e438f9225f9f5ed67f6ca P e2abad33dd7046b2bb589197ed70fa25: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260806 22:50:19.532219 5637 raft_consensus.cc:2229] T 736b36c0ea05421d9bc7a95f84c5dbeb P 589db803901b4d34aac64364c0164d6c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:50:19.532311 5637 raft_consensus.cc:2258] T 736b36c0ea05421d9bc7a95f84c5dbeb P 589db803901b4d34aac64364c0164d6c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:50:19.532897 5637 ts_tablet_manager.cc:1905] T 736b36c0ea05421d9bc7a95f84c5dbeb P 589db803901b4d34aac64364c0164d6c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:50:19.534389 5637 ts_tablet_manager.cc:1918] T 736b36c0ea05421d9bc7a95f84c5dbeb P 589db803901b4d34aac64364c0164d6c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:50:19.534471 5637 log.cc:1196] T 736b36c0ea05421d9bc7a95f84c5dbeb P 589db803901b4d34aac64364c0164d6c: Deleting WAL directory at /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056586788339-29547-0/minicluster-data/ts-0/wal/wals/736b36c0ea05421d9bc7a95f84c5dbeb
I20260806 22:50:19.534716 5637 ts_tablet_manager.cc:1939] T 736b36c0ea05421d9bc7a95f84c5dbeb P 589db803901b4d34aac64364c0164d6c: Deleting consensus metadata
I20260806 22:50:19.535084 5046 catalog_manager.cc:4841] TS 589db803901b4d34aac64364c0164d6c (127.28.218.193:42059): tablet 736b36c0ea05421d9bc7a95f84c5dbeb (table test-table [id=7e5bf214ab7243cd8f28eb992d028a3e]) successfully deleted
I20260806 22:50:19.539332 5635 ts_tablet_manager.cc:1905] T 736b36c0ea05421d9bc7a95f84c5dbeb P e2abad33dd7046b2bb589197ed70fa25: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:50:19.539806 5618 tablet_bootstrap.cc:492] T 8b7b7bdfb66e438f9225f9f5ed67f6ca P 589db803901b4d34aac64364c0164d6c: No bootstrap required, opened a new log
I20260806 22:50:19.539862 5618 ts_tablet_manager.cc:1397] T 8b7b7bdfb66e438f9225f9f5ed67f6ca P 589db803901b4d34aac64364c0164d6c: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.001s
I20260806 22:50:19.540015 5618 raft_consensus.cc:348] T 8b7b7bdfb66e438f9225f9f5ed67f6ca P 589db803901b4d34aac64364c0164d6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "589db803901b4d34aac64364c0164d6c" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 42059 } } peers { permanent_uuid: "e2abad33dd7046b2bb589197ed70fa25" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 41871 } } peers { permanent_uuid: "3b7f0599c31b4cd798417dc934125fd0" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 39813 } }
I20260806 22:50:19.540104 5618 raft_consensus.cc:374] T 8b7b7bdfb66e438f9225f9f5ed67f6ca P 589db803901b4d34aac64364c0164d6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:50:19.540154 5618 raft_consensus.cc:729] T 8b7b7bdfb66e438f9225f9f5ed67f6ca P 589db803901b4d34aac64364c0164d6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 589db803901b4d34aac64364c0164d6c, State: Initialized, Role: FOLLOWER
I20260806 22:50:19.540199 5618 consensus_queue.cc:260] T 8b7b7bdfb66e438f9225f9f5ed67f6ca P 589db803901b4d34aac64364c0164d6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "589db803901b4d34aac64364c0164d6c" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 42059 } } peers { permanent_uuid: "e2abad33dd7046b2bb589197ed70fa25" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 41871 } } peers { permanent_uuid: "3b7f0599c31b4cd798417dc934125fd0" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 39813 } }
I20260806 22:50:19.540305 5618 ts_tablet_manager.cc:1428] T 8b7b7bdfb66e438f9225f9f5ed67f6ca P 589db803901b4d34aac64364c0164d6c: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260806 22:50:19.541018 5635 ts_tablet_manager.cc:1918] T 736b36c0ea05421d9bc7a95f84c5dbeb P e2abad33dd7046b2bb589197ed70fa25: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:50:19.541074 5635 log.cc:1196] T 736b36c0ea05421d9bc7a95f84c5dbeb P e2abad33dd7046b2bb589197ed70fa25: Deleting WAL directory at /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056586788339-29547-0/minicluster-data/ts-1/wal/wals/736b36c0ea05421d9bc7a95f84c5dbeb
I20260806 22:50:19.541296 5635 ts_tablet_manager.cc:1939] T 736b36c0ea05421d9bc7a95f84c5dbeb P e2abad33dd7046b2bb589197ed70fa25: Deleting consensus metadata
I20260806 22:50:19.541877 5047 catalog_manager.cc:4841] TS e2abad33dd7046b2bb589197ed70fa25 (127.28.218.194:41871): tablet 736b36c0ea05421d9bc7a95f84c5dbeb (table test-table [id=7e5bf214ab7243cd8f28eb992d028a3e]) successfully deleted
W20260806 22:50:19.605692 5291 tablet.cc:2369] T 8b7b7bdfb66e438f9225f9f5ed67f6ca P 589db803901b4d34aac64364c0164d6c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:50:19.649224 5620 raft_consensus.cc:482] T 8b7b7bdfb66e438f9225f9f5ed67f6ca P 589db803901b4d34aac64364c0164d6c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:50:19.649325 5620 raft_consensus.cc:504] T 8b7b7bdfb66e438f9225f9f5ed67f6ca P 589db803901b4d34aac64364c0164d6c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "589db803901b4d34aac64364c0164d6c" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 42059 } } peers { permanent_uuid: "e2abad33dd7046b2bb589197ed70fa25" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 41871 } } peers { permanent_uuid: "3b7f0599c31b4cd798417dc934125fd0" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 39813 } }
I20260806 22:50:19.649713 5620 leader_election.cc:288] T 8b7b7bdfb66e438f9225f9f5ed67f6ca P 589db803901b4d34aac64364c0164d6c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e2abad33dd7046b2bb589197ed70fa25 (127.28.218.194:41871), 3b7f0599c31b4cd798417dc934125fd0 (127.28.218.195:39813)
I20260806 22:50:19.653877 5611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:19.649975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.193:37239 (local address 127.28.218.194:41871)
0806 22:50:19.650056 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:19.650059 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:19.650117 (+ 58us) server_negotiation.cc:407] Connection header received
0806 22:50:19.650200 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:19.650201 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:19.650211 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:19.650265 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:50:19.650470 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:19.651260 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:19.652871 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:19.653387 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:19.653463 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:19.653472 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:19.653771 (+ 299us) server_negotiation.cc:299] Negotiation successful
0806 22:50:19.653809 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 22:50:19.653877 5608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:19.650176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.193:44025 (local address 127.28.218.195:39813)
0806 22:50:19.650236 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:19.650238 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:19.650287 (+ 49us) server_negotiation.cc:407] Connection header received
0806 22:50:19.650364 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:19.650365 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:19.650377 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:19.650413 (+ 36us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:50:19.650554 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:19.651260 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:19.652871 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:19.653387 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:19.653463 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:19.653472 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:19.653771 (+ 299us) server_negotiation.cc:299] Negotiation successful
0806 22:50:19.653808 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260806 22:50:19.653944 5641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:19.649884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.194:41871 (local address 127.28.218.193:37239)
0806 22:50:19.650052 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:19.650065 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:19.650136 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:19.650295 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:19.650296 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:19.650299 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:50:19.650387 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:19.650389 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:19.651349 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:19.651351 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:19.652772 (+ 1421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:19.652775 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:19.653562 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:19.653563 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:19.653675 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:19.653683 (+ 8us) client_negotiation.cc:769] Sending connection context
0806 22:50:19.653750 (+ 67us) client_negotiation.cc:240] Negotiation successful
0806 22:50:19.653802 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"mutex_wait_us":90,"thread_start_us":92,"threads_started":1}
I20260806 22:50:19.654204 5389 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b7b7bdfb66e438f9225f9f5ed67f6ca" candidate_uuid: "589db803901b4d34aac64364c0164d6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2abad33dd7046b2bb589197ed70fa25" is_pre_election: true
I20260806 22:50:19.653944 5642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:19.650112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.195:39813 (local address 127.28.218.193:44025)
0806 22:50:19.650222 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:19.650233 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:19.650298 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:19.650434 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:19.650435 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:19.650436 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:50:19.650481 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:19.650483 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:19.651352 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:19.651352 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:19.652771 (+ 1419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:19.652775 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:19.653564 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:19.653564 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:19.653675 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:19.653683 (+ 8us) client_negotiation.cc:769] Sending connection context
0806 22:50:19.653750 (+ 67us) client_negotiation.cc:240] Negotiation successful
0806 22:50:19.653802 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"mutex_wait_us":91,"thread_start_us":43,"threads_started":1}
I20260806 22:50:19.654299 5389 raft_consensus.cc:2454] T 8b7b7bdfb66e438f9225f9f5ed67f6ca P e2abad33dd7046b2bb589197ed70fa25 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 589db803901b4d34aac64364c0164d6c in term 0.
I20260806 22:50:19.654472 5168 leader_election.cc:302] T 8b7b7bdfb66e438f9225f9f5ed67f6ca P 589db803901b4d34aac64364c0164d6c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 589db803901b4d34aac64364c0164d6c, e2abad33dd7046b2bb589197ed70fa25; no voters:
I20260806 22:50:19.654553 5532 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b7b7bdfb66e438f9225f9f5ed67f6ca" candidate_uuid: "589db803901b4d34aac64364c0164d6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b7f0599c31b4cd798417dc934125fd0" is_pre_election: true
I20260806 22:50:19.654639 5620 raft_consensus.cc:2781] T 8b7b7bdfb66e438f9225f9f5ed67f6ca P 589db803901b4d34aac64364c0164d6c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:50:19.654680 5532 raft_consensus.cc:2454] T 8b7b7bdfb66e438f9225f9f5ed67f6ca P 3b7f0599c31b4cd798417dc934125fd0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 589db803901b4d34aac64364c0164d6c in term 0.
I20260806 22:50:19.654693 5620 raft_consensus.cc:482] T 8b7b7bdfb66e438f9225f9f5ed67f6ca P 589db803901b4d34aac64364c0164d6c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:50:19.654719 5620 raft_consensus.cc:3037] T 8b7b7bdfb66e438f9225f9f5ed67f6ca P 589db803901b4d34aac64364c0164d6c [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:50:19.655448 5620 raft_consensus.cc:504] T 8b7b7bdfb66e438f9225f9f5ed67f6ca P 589db803901b4d34aac64364c0164d6c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "589db803901b4d34aac64364c0164d6c" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 42059 } } peers { permanent_uuid: "e2abad33dd7046b2bb589197ed70fa25" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 41871 } } peers { permanent_uuid: "3b7f0599c31b4cd798417dc934125fd0" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 39813 } }
I20260806 22:50:19.655584 5620 leader_election.cc:288] T 8b7b7bdfb66e438f9225f9f5ed67f6ca P 589db803901b4d34aac64364c0164d6c [CANDIDATE]: Term 1 election: Requested vote from peers e2abad33dd7046b2bb589197ed70fa25 (127.28.218.194:41871), 3b7f0599c31b4cd798417dc934125fd0 (127.28.218.195:39813)
I20260806 22:50:19.655741 5389 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b7b7bdfb66e438f9225f9f5ed67f6ca" candidate_uuid: "589db803901b4d34aac64364c0164d6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2abad33dd7046b2bb589197ed70fa25"
I20260806 22:50:19.655762 5532 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b7b7bdfb66e438f9225f9f5ed67f6ca" candidate_uuid: "589db803901b4d34aac64364c0164d6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b7f0599c31b4cd798417dc934125fd0"
I20260806 22:50:19.655825 5532 raft_consensus.cc:3037] T 8b7b7bdfb66e438f9225f9f5ed67f6ca P 3b7f0599c31b4cd798417dc934125fd0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:50:19.655825 5389 raft_consensus.cc:3037] T 8b7b7bdfb66e438f9225f9f5ed67f6ca P e2abad33dd7046b2bb589197ed70fa25 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:50:19.656626 5532 raft_consensus.cc:2454] T 8b7b7bdfb66e438f9225f9f5ed67f6ca P 3b7f0599c31b4cd798417dc934125fd0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 589db803901b4d34aac64364c0164d6c in term 1.
I20260806 22:50:19.656623 5389 raft_consensus.cc:2454] T 8b7b7bdfb66e438f9225f9f5ed67f6ca P e2abad33dd7046b2bb589197ed70fa25 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 589db803901b4d34aac64364c0164d6c in term 1.
I20260806 22:50:19.656810 5168 leader_election.cc:302] T 8b7b7bdfb66e438f9225f9f5ed67f6ca P 589db803901b4d34aac64364c0164d6c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b7f0599c31b4cd798417dc934125fd0, 589db803901b4d34aac64364c0164d6c; no voters:
I20260806 22:50:19.656889 5620 raft_consensus.cc:2781] T 8b7b7bdfb66e438f9225f9f5ed67f6ca P 589db803901b4d34aac64364c0164d6c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:50:19.657001 5620 raft_consensus.cc:686] T 8b7b7bdfb66e438f9225f9f5ed67f6ca P 589db803901b4d34aac64364c0164d6c [term 1 LEADER]: Becoming Leader. State: Replica: 589db803901b4d34aac64364c0164d6c, State: Running, Role: LEADER
I20260806 22:50:19.657078 5620 consensus_queue.cc:237] T 8b7b7bdfb66e438f9225f9f5ed67f6ca P 589db803901b4d34aac64364c0164d6c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "589db803901b4d34aac64364c0164d6c" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 42059 } } peers { permanent_uuid: "e2abad33dd7046b2bb589197ed70fa25" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 41871 } } peers { permanent_uuid: "3b7f0599c31b4cd798417dc934125fd0" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 39813 } }
I20260806 22:50:19.657766 5060 catalog_manager.cc:5495] T 8b7b7bdfb66e438f9225f9f5ed67f6ca P 589db803901b4d34aac64364c0164d6c reported cstate change: term changed from 0 to 1, leader changed from <none> to 589db803901b4d34aac64364c0164d6c (127.28.218.193). New cstate: current_term: 1 leader_uuid: "589db803901b4d34aac64364c0164d6c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "589db803901b4d34aac64364c0164d6c" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 42059 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e2abad33dd7046b2bb589197ed70fa25" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 41871 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b7f0599c31b4cd798417dc934125fd0" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 39813 } health_report { overall_health: UNKNOWN } } }
W20260806 22:50:19.659989 5582 tablet.cc:2369] T 8b7b7bdfb66e438f9225f9f5ed67f6ca P 3b7f0599c31b4cd798417dc934125fd0: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 06 22:50:19 dist-test-slave-1xrh krb5kdc[5015](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056618, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.218.254@KRBTEST.COM for oryx/127.28.218.254@KRBTEST.COM
I20260806 22:50:19.706132 5123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:19.699739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40678 (local address 127.28.218.254:33353)
0806 22:50:19.699820 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:19.699822 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:19.699934 (+ 112us) server_negotiation.cc:407] Connection header received
0806 22:50:19.700010 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:19.700011 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:19.700020 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:19.700070 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:19.700499 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:19.701063 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:19.701735 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:19.701919 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:19.701990 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:19.703947 (+ 1957us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:50:19.703948 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:50:19.703949 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:50:19.703956 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:50:19.705088 (+ 1132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:19.705408 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:19.705410 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:19.705410 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:19.705462 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:19.705642 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:19.705643 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:19.705643 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:19.705734 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:50:19.705798 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:19.705943 (+ 145us) server_negotiation.cc:299] Negotiation successful
0806 22:50:19.706035 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 22:50:19.706202 5651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:19.699674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:33353 (local address 127.0.0.1:40678)
0806 22:50:19.699842 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:19.699858 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:19.699949 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:19.700077 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:19.700078 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:19.700362 (+ 284us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:19.700401 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:19.700403 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:19.701143 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:19.701144 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:19.701653 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:19.701655 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:19.701988 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:19.701989 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:19.702052 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:19.702592 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:50:19.702596 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:19.703835 (+ 1239us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:50:19.705163 (+ 1328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:19.705170 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:19.705172 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:19.705342 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:19.705522 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:19.705523 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:19.705523 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:19.705577 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:19.705802 (+ 225us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:50:19.705809 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:50:19.705846 (+ 37us) client_negotiation.cc:769] Sending connection context
0806 22:50:19.705921 (+ 75us) client_negotiation.cc:240] Negotiation successful
0806 22:50:19.706088 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"mutex_wait_us":79,"thread_start_us":95,"threads_started":1}
I20260806 22:50:19.708017 5060 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.28.218.254@KRBTEST.COM'} at 127.0.0.1:40678:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 22:50:19.708223 5060 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:50:19.709502 5366 tablet_service.cc:1467] Processing CreateTablet for tablet cfd6766325ad49ba9dea9219408cd2da (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=9646dc34e13d4a81b63277cfb56719c2]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 22:50:19.709502 5496 tablet_service.cc:1467] Processing CreateTablet for tablet cfd6766325ad49ba9dea9219408cd2da (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=9646dc34e13d4a81b63277cfb56719c2]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 22:50:19.709470 5217 tablet_service.cc:1467] Processing CreateTablet for tablet cfd6766325ad49ba9dea9219408cd2da (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=9646dc34e13d4a81b63277cfb56719c2]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 22:50:19.709674 5366 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cfd6766325ad49ba9dea9219408cd2da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:50:19.709656 5496 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cfd6766325ad49ba9dea9219408cd2da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:50:19.709656 5217 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cfd6766325ad49ba9dea9219408cd2da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:50:19.710798 5612 tablet_bootstrap.cc:492] T cfd6766325ad49ba9dea9219408cd2da P e2abad33dd7046b2bb589197ed70fa25: Bootstrap starting.
I20260806 22:50:19.711105 5613 tablet_bootstrap.cc:492] T cfd6766325ad49ba9dea9219408cd2da P 3b7f0599c31b4cd798417dc934125fd0: Bootstrap starting.
I20260806 22:50:19.711143 5618 tablet_bootstrap.cc:492] T cfd6766325ad49ba9dea9219408cd2da P 589db803901b4d34aac64364c0164d6c: Bootstrap starting.
I20260806 22:50:19.711253 5612 tablet_bootstrap.cc:654] T cfd6766325ad49ba9dea9219408cd2da P e2abad33dd7046b2bb589197ed70fa25: Neither blocks nor log segments found. Creating new log.
I20260806 22:50:19.711478 5613 tablet_bootstrap.cc:654] T cfd6766325ad49ba9dea9219408cd2da P 3b7f0599c31b4cd798417dc934125fd0: Neither blocks nor log segments found. Creating new log.
I20260806 22:50:19.711597 5618 tablet_bootstrap.cc:654] T cfd6766325ad49ba9dea9219408cd2da P 589db803901b4d34aac64364c0164d6c: Neither blocks nor log segments found. Creating new log.
I20260806 22:50:19.712097 5613 tablet_bootstrap.cc:492] T cfd6766325ad49ba9dea9219408cd2da P 3b7f0599c31b4cd798417dc934125fd0: No bootstrap required, opened a new log
I20260806 22:50:19.712150 5613 ts_tablet_manager.cc:1397] T cfd6766325ad49ba9dea9219408cd2da P 3b7f0599c31b4cd798417dc934125fd0: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 22:50:19.712286 5613 raft_consensus.cc:348] T cfd6766325ad49ba9dea9219408cd2da P 3b7f0599c31b4cd798417dc934125fd0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "589db803901b4d34aac64364c0164d6c" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 42059 } } peers { permanent_uuid: "e2abad33dd7046b2bb589197ed70fa25" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 41871 } } peers { permanent_uuid: "3b7f0599c31b4cd798417dc934125fd0" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 39813 } }
I20260806 22:50:19.712364 5613 raft_consensus.cc:374] T cfd6766325ad49ba9dea9219408cd2da P 3b7f0599c31b4cd798417dc934125fd0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:50:19.712390 5613 raft_consensus.cc:729] T cfd6766325ad49ba9dea9219408cd2da P 3b7f0599c31b4cd798417dc934125fd0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b7f0599c31b4cd798417dc934125fd0, State: Initialized, Role: FOLLOWER
I20260806 22:50:19.712436 5613 consensus_queue.cc:260] T cfd6766325ad49ba9dea9219408cd2da P 3b7f0599c31b4cd798417dc934125fd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "589db803901b4d34aac64364c0164d6c" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 42059 } } peers { permanent_uuid: "e2abad33dd7046b2bb589197ed70fa25" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 41871 } } peers { permanent_uuid: "3b7f0599c31b4cd798417dc934125fd0" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 39813 } }
I20260806 22:50:19.712493 5618 tablet_bootstrap.cc:492] T cfd6766325ad49ba9dea9219408cd2da P 589db803901b4d34aac64364c0164d6c: No bootstrap required, opened a new log
I20260806 22:50:19.712556 5618 ts_tablet_manager.cc:1397] T cfd6766325ad49ba9dea9219408cd2da P 589db803901b4d34aac64364c0164d6c: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 22:50:19.712584 5613 ts_tablet_manager.cc:1428] T cfd6766325ad49ba9dea9219408cd2da P 3b7f0599c31b4cd798417dc934125fd0: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260806 22:50:19.712633 5612 tablet_bootstrap.cc:492] T cfd6766325ad49ba9dea9219408cd2da P e2abad33dd7046b2bb589197ed70fa25: No bootstrap required, opened a new log
I20260806 22:50:19.712683 5612 ts_tablet_manager.cc:1397] T cfd6766325ad49ba9dea9219408cd2da P e2abad33dd7046b2bb589197ed70fa25: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260806 22:50:19.712679 5622 tablet_replica.cc:439] T cfd6766325ad49ba9dea9219408cd2da P 3b7f0599c31b4cd798417dc934125fd0: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "589db803901b4d34aac64364c0164d6c" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 42059 } } peers { permanent_uuid: "e2abad33dd7046b2bb589197ed70fa25" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 41871 } } peers { permanent_uuid: "3b7f0599c31b4cd798417dc934125fd0" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 39813 } } }
I20260806 22:50:19.712695 5618 raft_consensus.cc:348] T cfd6766325ad49ba9dea9219408cd2da P 589db803901b4d34aac64364c0164d6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "589db803901b4d34aac64364c0164d6c" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 42059 } } peers { permanent_uuid: "e2abad33dd7046b2bb589197ed70fa25" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 41871 } } peers { permanent_uuid: "3b7f0599c31b4cd798417dc934125fd0" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 39813 } }
I20260806 22:50:19.712764 5618 raft_consensus.cc:374] T cfd6766325ad49ba9dea9219408cd2da P 589db803901b4d34aac64364c0164d6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:50:19.712767 5622 tablet_replica.cc:442] T cfd6766325ad49ba9dea9219408cd2da P 3b7f0599c31b4cd798417dc934125fd0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:50:19.712790 5618 raft_consensus.cc:729] T cfd6766325ad49ba9dea9219408cd2da P 589db803901b4d34aac64364c0164d6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 589db803901b4d34aac64364c0164d6c, State: Initialized, Role: FOLLOWER
I20260806 22:50:19.712828 5618 consensus_queue.cc:260] T cfd6766325ad49ba9dea9219408cd2da P 589db803901b4d34aac64364c0164d6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "589db803901b4d34aac64364c0164d6c" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 42059 } } peers { permanent_uuid: "e2abad33dd7046b2bb589197ed70fa25" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 41871 } } peers { permanent_uuid: "3b7f0599c31b4cd798417dc934125fd0" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 39813 } }
I20260806 22:50:19.712832 5612 raft_consensus.cc:348] T cfd6766325ad49ba9dea9219408cd2da P e2abad33dd7046b2bb589197ed70fa25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "589db803901b4d34aac64364c0164d6c" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 42059 } } peers { permanent_uuid: "e2abad33dd7046b2bb589197ed70fa25" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 41871 } } peers { permanent_uuid: "3b7f0599c31b4cd798417dc934125fd0" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 39813 } }
I20260806 22:50:19.712898 5612 raft_consensus.cc:374] T cfd6766325ad49ba9dea9219408cd2da P e2abad33dd7046b2bb589197ed70fa25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:50:19.712914 5618 ts_tablet_manager.cc:1428] T cfd6766325ad49ba9dea9219408cd2da P 589db803901b4d34aac64364c0164d6c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:19.712922 5612 raft_consensus.cc:729] T cfd6766325ad49ba9dea9219408cd2da P e2abad33dd7046b2bb589197ed70fa25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2abad33dd7046b2bb589197ed70fa25, State: Initialized, Role: FOLLOWER
I20260806 22:50:19.712967 5612 consensus_queue.cc:260] T cfd6766325ad49ba9dea9219408cd2da P e2abad33dd7046b2bb589197ed70fa25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "589db803901b4d34aac64364c0164d6c" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 42059 } } peers { permanent_uuid: "e2abad33dd7046b2bb589197ed70fa25" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 41871 } } peers { permanent_uuid: "3b7f0599c31b4cd798417dc934125fd0" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 39813 } }
I20260806 22:50:19.713078 5612 ts_tablet_manager.cc:1428] T cfd6766325ad49ba9dea9219408cd2da P e2abad33dd7046b2bb589197ed70fa25: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260806 22:50:19.713188 5620 tablet_replica.cc:439] T cfd6766325ad49ba9dea9219408cd2da P 589db803901b4d34aac64364c0164d6c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "589db803901b4d34aac64364c0164d6c" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 42059 } } peers { permanent_uuid: "e2abad33dd7046b2bb589197ed70fa25" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 41871 } } peers { permanent_uuid: "3b7f0599c31b4cd798417dc934125fd0" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 39813 } } }
I20260806 22:50:19.713263 5620 tablet_replica.cc:442] T cfd6766325ad49ba9dea9219408cd2da P 589db803901b4d34aac64364c0164d6c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:50:19.713399 5617 tablet_replica.cc:439] T cfd6766325ad49ba9dea9219408cd2da P e2abad33dd7046b2bb589197ed70fa25: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "589db803901b4d34aac64364c0164d6c" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 42059 } } peers { permanent_uuid: "e2abad33dd7046b2bb589197ed70fa25" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 41871 } } peers { permanent_uuid: "3b7f0599c31b4cd798417dc934125fd0" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 39813 } } }
I20260806 22:50:19.713488 5617 tablet_replica.cc:442] T cfd6766325ad49ba9dea9219408cd2da P e2abad33dd7046b2bb589197ed70fa25: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:50:19.959674 5620 raft_consensus.cc:482] T cfd6766325ad49ba9dea9219408cd2da P 589db803901b4d34aac64364c0164d6c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:50:19.959795 5620 raft_consensus.cc:504] T cfd6766325ad49ba9dea9219408cd2da P 589db803901b4d34aac64364c0164d6c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "589db803901b4d34aac64364c0164d6c" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 42059 } } peers { permanent_uuid: "e2abad33dd7046b2bb589197ed70fa25" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 41871 } } peers { permanent_uuid: "3b7f0599c31b4cd798417dc934125fd0" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 39813 } }
I20260806 22:50:19.959946 5620 leader_election.cc:288] T cfd6766325ad49ba9dea9219408cd2da P 589db803901b4d34aac64364c0164d6c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e2abad33dd7046b2bb589197ed70fa25 (127.28.218.194:41871), 3b7f0599c31b4cd798417dc934125fd0 (127.28.218.195:39813)
I20260806 22:50:19.960206 5532 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfd6766325ad49ba9dea9219408cd2da" candidate_uuid: "589db803901b4d34aac64364c0164d6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b7f0599c31b4cd798417dc934125fd0" is_pre_election: true
I20260806 22:50:19.960201 5389 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfd6766325ad49ba9dea9219408cd2da" candidate_uuid: "589db803901b4d34aac64364c0164d6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2abad33dd7046b2bb589197ed70fa25" is_pre_election: true
I20260806 22:50:19.960296 5389 raft_consensus.cc:2454] T cfd6766325ad49ba9dea9219408cd2da P e2abad33dd7046b2bb589197ed70fa25 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 589db803901b4d34aac64364c0164d6c in term 0.
I20260806 22:50:19.960317 5532 raft_consensus.cc:2454] T cfd6766325ad49ba9dea9219408cd2da P 3b7f0599c31b4cd798417dc934125fd0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 589db803901b4d34aac64364c0164d6c in term 0.
I20260806 22:50:19.960474 5168 leader_election.cc:302] T cfd6766325ad49ba9dea9219408cd2da P 589db803901b4d34aac64364c0164d6c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 589db803901b4d34aac64364c0164d6c, e2abad33dd7046b2bb589197ed70fa25; no voters:
I20260806 22:50:19.960595 5620 raft_consensus.cc:2781] T cfd6766325ad49ba9dea9219408cd2da P 589db803901b4d34aac64364c0164d6c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:50:19.960649 5620 raft_consensus.cc:482] T cfd6766325ad49ba9dea9219408cd2da P 589db803901b4d34aac64364c0164d6c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:50:19.960671 5620 raft_consensus.cc:3037] T cfd6766325ad49ba9dea9219408cd2da P 589db803901b4d34aac64364c0164d6c [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:50:19.961364 5620 raft_consensus.cc:504] T cfd6766325ad49ba9dea9219408cd2da P 589db803901b4d34aac64364c0164d6c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "589db803901b4d34aac64364c0164d6c" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 42059 } } peers { permanent_uuid: "e2abad33dd7046b2bb589197ed70fa25" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 41871 } } peers { permanent_uuid: "3b7f0599c31b4cd798417dc934125fd0" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 39813 } }
I20260806 22:50:19.961494 5620 leader_election.cc:288] T cfd6766325ad49ba9dea9219408cd2da P 589db803901b4d34aac64364c0164d6c [CANDIDATE]: Term 1 election: Requested vote from peers e2abad33dd7046b2bb589197ed70fa25 (127.28.218.194:41871), 3b7f0599c31b4cd798417dc934125fd0 (127.28.218.195:39813)
I20260806 22:50:19.961704 5389 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfd6766325ad49ba9dea9219408cd2da" candidate_uuid: "589db803901b4d34aac64364c0164d6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2abad33dd7046b2bb589197ed70fa25"
I20260806 22:50:19.961725 5532 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfd6766325ad49ba9dea9219408cd2da" candidate_uuid: "589db803901b4d34aac64364c0164d6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b7f0599c31b4cd798417dc934125fd0"
I20260806 22:50:19.961781 5389 raft_consensus.cc:3037] T cfd6766325ad49ba9dea9219408cd2da P e2abad33dd7046b2bb589197ed70fa25 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:50:19.961812 5532 raft_consensus.cc:3037] T cfd6766325ad49ba9dea9219408cd2da P 3b7f0599c31b4cd798417dc934125fd0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:50:19.962374 5389 raft_consensus.cc:2454] T cfd6766325ad49ba9dea9219408cd2da P e2abad33dd7046b2bb589197ed70fa25 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 589db803901b4d34aac64364c0164d6c in term 1.
I20260806 22:50:19.962556 5168 leader_election.cc:302] T cfd6766325ad49ba9dea9219408cd2da P 589db803901b4d34aac64364c0164d6c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 589db803901b4d34aac64364c0164d6c, e2abad33dd7046b2bb589197ed70fa25; no voters:
I20260806 22:50:19.962671 5620 raft_consensus.cc:2781] T cfd6766325ad49ba9dea9219408cd2da P 589db803901b4d34aac64364c0164d6c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:50:19.962733 5532 raft_consensus.cc:2454] T cfd6766325ad49ba9dea9219408cd2da P 3b7f0599c31b4cd798417dc934125fd0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 589db803901b4d34aac64364c0164d6c in term 1.
I20260806 22:50:19.962745 5620 raft_consensus.cc:686] T cfd6766325ad49ba9dea9219408cd2da P 589db803901b4d34aac64364c0164d6c [term 1 LEADER]: Becoming Leader. State: Replica: 589db803901b4d34aac64364c0164d6c, State: Running, Role: LEADER
I20260806 22:50:19.962801 5620 consensus_queue.cc:237] T cfd6766325ad49ba9dea9219408cd2da P 589db803901b4d34aac64364c0164d6c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "589db803901b4d34aac64364c0164d6c" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 42059 } } peers { permanent_uuid: "e2abad33dd7046b2bb589197ed70fa25" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 41871 } } peers { permanent_uuid: "3b7f0599c31b4cd798417dc934125fd0" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 39813 } }
I20260806 22:50:19.963202 5643 tablet_replica.cc:439] T cfd6766325ad49ba9dea9219408cd2da P 589db803901b4d34aac64364c0164d6c: TxnStatusTablet state changed. Reason: New leader 589db803901b4d34aac64364c0164d6c. Latest consensus state: current_term: 1 leader_uuid: "589db803901b4d34aac64364c0164d6c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "589db803901b4d34aac64364c0164d6c" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 42059 } } peers { permanent_uuid: "e2abad33dd7046b2bb589197ed70fa25" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 41871 } } peers { permanent_uuid: "3b7f0599c31b4cd798417dc934125fd0" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 39813 } } }
I20260806 22:50:19.963320 5643 tablet_replica.cc:442] T cfd6766325ad49ba9dea9219408cd2da P 589db803901b4d34aac64364c0164d6c: This TxnStatusTablet replica's current role is: LEADER
I20260806 22:50:19.963454 5060 catalog_manager.cc:5495] T cfd6766325ad49ba9dea9219408cd2da P 589db803901b4d34aac64364c0164d6c reported cstate change: term changed from 0 to 1, leader changed from <none> to 589db803901b4d34aac64364c0164d6c (127.28.218.193). New cstate: current_term: 1 leader_uuid: "589db803901b4d34aac64364c0164d6c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "589db803901b4d34aac64364c0164d6c" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 42059 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e2abad33dd7046b2bb589197ed70fa25" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 41871 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b7f0599c31b4cd798417dc934125fd0" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 39813 } health_report { overall_health: UNKNOWN } } }
I20260806 22:50:19.963518 5656 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 22:50:19.963629 5656 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 22:50:19.996438 5651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:19.993484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.193:42059 (local address 127.0.0.1:44898)
0806 22:50:19.993563 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:19.993575 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:19.993639 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:19.993843 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:19.993844 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:19.993845 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 22:50:19.993889 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:19.993892 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:19.994827 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:19.994828 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:19.995499 (+ 671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:19.995502 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:19.995967 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:19.995967 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:19.996014 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:19.996028 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 22:50:19.996306 (+ 278us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 22:50:19.996312 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 22:50:19.996343 (+ 31us) client_negotiation.cc:240] Negotiation successful
0806 22:50:19.996380 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260806 22:50:19.996452 5609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:19.993552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44898 (local address 127.28.218.193:42059)
0806 22:50:19.993595 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:19.993597 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:19.993607 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:50:19.993718 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:19.993719 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:19.993738 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:19.993836 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:50:19.993963 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:19.994748 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:19.995579 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:19.995888 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:19.995965 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:19.996120 (+ 155us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:50:19.996240 (+ 120us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:50:19.996314 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:19.996353 (+ 39us) server_negotiation.cc:299] Negotiation successful
0806 22:50:19.996373 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 22:50:19.997712 5389 raft_consensus.cc:1261] T cfd6766325ad49ba9dea9219408cd2da P e2abad33dd7046b2bb589197ed70fa25 [term 1 FOLLOWER]: Refusing update from remote peer 589db803901b4d34aac64364c0164d6c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:50:19.997731 5532 raft_consensus.cc:1261] T cfd6766325ad49ba9dea9219408cd2da P 3b7f0599c31b4cd798417dc934125fd0 [term 1 FOLLOWER]: Refusing update from remote peer 589db803901b4d34aac64364c0164d6c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:50:19.997999 5620 consensus_queue.cc:1035] T cfd6766325ad49ba9dea9219408cd2da P 589db803901b4d34aac64364c0164d6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b7f0599c31b4cd798417dc934125fd0" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 39813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:50:19.998128 5643 consensus_queue.cc:1035] T cfd6766325ad49ba9dea9219408cd2da P 589db803901b4d34aac64364c0164d6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2abad33dd7046b2bb589197ed70fa25" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 41871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:50:19.998826 5617 tablet_replica.cc:439] T cfd6766325ad49ba9dea9219408cd2da P e2abad33dd7046b2bb589197ed70fa25: TxnStatusTablet state changed. Reason: New leader 589db803901b4d34aac64364c0164d6c. Latest consensus state: current_term: 1 leader_uuid: "589db803901b4d34aac64364c0164d6c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "589db803901b4d34aac64364c0164d6c" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 42059 } } peers { permanent_uuid: "e2abad33dd7046b2bb589197ed70fa25" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 41871 } } peers { permanent_uuid: "3b7f0599c31b4cd798417dc934125fd0" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 39813 } } }
I20260806 22:50:19.998917 5617 tablet_replica.cc:442] T cfd6766325ad49ba9dea9219408cd2da P e2abad33dd7046b2bb589197ed70fa25: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:50:19.998991 5622 tablet_replica.cc:439] T cfd6766325ad49ba9dea9219408cd2da P 3b7f0599c31b4cd798417dc934125fd0: TxnStatusTablet state changed. Reason: New leader 589db803901b4d34aac64364c0164d6c. Latest consensus state: current_term: 1 leader_uuid: "589db803901b4d34aac64364c0164d6c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "589db803901b4d34aac64364c0164d6c" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 42059 } } peers { permanent_uuid: "e2abad33dd7046b2bb589197ed70fa25" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 41871 } } peers { permanent_uuid: "3b7f0599c31b4cd798417dc934125fd0" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 39813 } } }
I20260806 22:50:19.999079 5622 tablet_replica.cc:442] T cfd6766325ad49ba9dea9219408cd2da P 3b7f0599c31b4cd798417dc934125fd0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:50:19.999365 5643 tablet_replica.cc:439] T cfd6766325ad49ba9dea9219408cd2da P 589db803901b4d34aac64364c0164d6c: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "589db803901b4d34aac64364c0164d6c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "589db803901b4d34aac64364c0164d6c" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 42059 } } peers { permanent_uuid: "e2abad33dd7046b2bb589197ed70fa25" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 41871 } } peers { permanent_uuid: "3b7f0599c31b4cd798417dc934125fd0" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 39813 } } }
I20260806 22:50:19.999466 5643 tablet_replica.cc:442] T cfd6766325ad49ba9dea9219408cd2da P 589db803901b4d34aac64364c0164d6c: This TxnStatusTablet replica's current role is: LEADER
I20260806 22:50:20.000142 5643 tablet_replica.cc:439] T cfd6766325ad49ba9dea9219408cd2da P 589db803901b4d34aac64364c0164d6c: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "589db803901b4d34aac64364c0164d6c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "589db803901b4d34aac64364c0164d6c" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 42059 } } peers { permanent_uuid: "e2abad33dd7046b2bb589197ed70fa25" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 41871 } } peers { permanent_uuid: "3b7f0599c31b4cd798417dc934125fd0" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 39813 } } }
I20260806 22:50:20.000218 5643 tablet_replica.cc:442] T cfd6766325ad49ba9dea9219408cd2da P 589db803901b4d34aac64364c0164d6c: This TxnStatusTablet replica's current role is: LEADER
I20260806 22:50:20.000600 5617 tablet_replica.cc:439] T cfd6766325ad49ba9dea9219408cd2da P e2abad33dd7046b2bb589197ed70fa25: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "589db803901b4d34aac64364c0164d6c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "589db803901b4d34aac64364c0164d6c" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 42059 } } peers { permanent_uuid: "e2abad33dd7046b2bb589197ed70fa25" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 41871 } } peers { permanent_uuid: "3b7f0599c31b4cd798417dc934125fd0" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 39813 } } }
I20260806 22:50:20.000686 5617 tablet_replica.cc:442] T cfd6766325ad49ba9dea9219408cd2da P e2abad33dd7046b2bb589197ed70fa25: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:50:20.000756 5622 tablet_replica.cc:439] T cfd6766325ad49ba9dea9219408cd2da P 3b7f0599c31b4cd798417dc934125fd0: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "589db803901b4d34aac64364c0164d6c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "589db803901b4d34aac64364c0164d6c" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 42059 } } peers { permanent_uuid: "e2abad33dd7046b2bb589197ed70fa25" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 41871 } } peers { permanent_uuid: "3b7f0599c31b4cd798417dc934125fd0" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 39813 } } }
I20260806 22:50:20.000833 5622 tablet_replica.cc:442] T cfd6766325ad49ba9dea9219408cd2da P 3b7f0599c31b4cd798417dc934125fd0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:50:20.004220 5609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:20.001564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44906 (local address 127.28.218.193:42059)
0806 22:50:20.001595 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:20.001597 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:20.001635 (+ 38us) server_negotiation.cc:407] Connection header received
0806 22:50:20.001708 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:20.001709 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:20.001717 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:20.001772 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:50:20.001850 (+ 78us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:20.002567 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:20.003468 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:20.003757 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:20.003836 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:20.003991 (+ 155us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:50:20.004030 (+ 39us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:50:20.004085 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:20.004131 (+ 46us) server_negotiation.cc:299] Negotiation successful
0806 22:50:20.004148 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260806 22:50:20.009285 5609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:20.006713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.193:54689 (local address 127.28.218.193:42059)
0806 22:50:20.006749 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:20.006751 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:20.006882 (+ 131us) server_negotiation.cc:407] Connection header received
0806 22:50:20.006907 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:20.006908 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:20.006917 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:20.006979 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:50:20.007088 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:20.007766 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:20.008537 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:20.008848 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:20.008918 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:20.009062 (+ 144us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:50:20.009093 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:50:20.009149 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:20.009188 (+ 39us) server_negotiation.cc:299] Negotiation successful
0806 22:50:20.009204 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 22:50:20.009294 5664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:20.006663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.193:42059 (local address 127.28.218.193:54689)
0806 22:50:20.006815 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:20.006823 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:20.006873 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:20.006980 (+ 107us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:20.006981 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:20.006982 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 22:50:20.007034 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:20.007035 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:20.007839 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:20.007839 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:20.008459 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:20.008461 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:20.008929 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:20.008929 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:20.008976 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:20.008988 (+ 12us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 22:50:20.009149 (+ 161us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 22:50:20.009150 (+ 1us) client_negotiation.cc:769] Sending connection context
0806 22:50:20.009177 (+ 27us) client_negotiation.cc:240] Negotiation successful
0806 22:50:20.009208 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":63,"threads_started":1}
I20260806 22:50:20.011303 5532 raft_consensus.cc:1261] T 8b7b7bdfb66e438f9225f9f5ed67f6ca P 3b7f0599c31b4cd798417dc934125fd0 [term 1 FOLLOWER]: Refusing update from remote peer 589db803901b4d34aac64364c0164d6c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:50:20.011317 5389 raft_consensus.cc:1261] T 8b7b7bdfb66e438f9225f9f5ed67f6ca P e2abad33dd7046b2bb589197ed70fa25 [term 1 FOLLOWER]: Refusing update from remote peer 589db803901b4d34aac64364c0164d6c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:50:20.011511 5665 consensus_queue.cc:1035] T 8b7b7bdfb66e438f9225f9f5ed67f6ca P 589db803901b4d34aac64364c0164d6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b7f0599c31b4cd798417dc934125fd0" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 39813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:50:20.011626 5643 consensus_queue.cc:1035] T 8b7b7bdfb66e438f9225f9f5ed67f6ca P 589db803901b4d34aac64364c0164d6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2abad33dd7046b2bb589197ed70fa25" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 41871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:50:20.013213 5627 mvcc.cc:205] Tried to move back new op lower bound from 7315687915565019136 to 7315687914115973120. Current Snapshot: MvccSnapshot[applied={T|T < 7315687915565019136}]
I20260806 22:50:20.025178 5060 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40676:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:50:20.025277 5060 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40676:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:50:20.025854 5060 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ecdca47ce9f544afa6f7395b0ad3445e: Sending DeleteTablet for 3 replicas of tablet 8b7b7bdfb66e438f9225f9f5ed67f6ca
I20260806 22:50:20.026111 5366 tablet_service.cc:1514] Processing DeleteTablet for tablet 8b7b7bdfb66e438f9225f9f5ed67f6ca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:50:20 UTC) from {username='oryx', principal='oryx/127.28.218.254@KRBTEST.COM'} at 127.0.0.1:47640
I20260806 22:50:20.026121 5496 tablet_service.cc:1514] Processing DeleteTablet for tablet 8b7b7bdfb66e438f9225f9f5ed67f6ca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:50:20 UTC) from {username='oryx', principal='oryx/127.28.218.254@KRBTEST.COM'} at 127.0.0.1:33464
I20260806 22:50:20.026206 5635 tablet_replica.cc:330] T 8b7b7bdfb66e438f9225f9f5ed67f6ca P e2abad33dd7046b2bb589197ed70fa25: stopping tablet replica
I20260806 22:50:20.026211 5636 tablet_replica.cc:330] T 8b7b7bdfb66e438f9225f9f5ed67f6ca P 3b7f0599c31b4cd798417dc934125fd0: stopping tablet replica
I20260806 22:50:20.026257 5636 raft_consensus.cc:2229] T 8b7b7bdfb66e438f9225f9f5ed67f6ca P 3b7f0599c31b4cd798417dc934125fd0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:50:20.026257 5635 raft_consensus.cc:2229] T 8b7b7bdfb66e438f9225f9f5ed67f6ca P e2abad33dd7046b2bb589197ed70fa25 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:50:20.026293 5636 raft_consensus.cc:2258] T 8b7b7bdfb66e438f9225f9f5ed67f6ca P 3b7f0599c31b4cd798417dc934125fd0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:50:20.026311 5635 raft_consensus.cc:2258] T 8b7b7bdfb66e438f9225f9f5ed67f6ca P e2abad33dd7046b2bb589197ed70fa25 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:50:20.026501 5217 tablet_service.cc:1514] Processing DeleteTablet for tablet 8b7b7bdfb66e438f9225f9f5ed67f6ca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:50:20 UTC) from {username='oryx', principal='oryx/127.28.218.254@KRBTEST.COM'} at 127.0.0.1:44890
I20260806 22:50:20.026579 5637 tablet_replica.cc:330] T 8b7b7bdfb66e438f9225f9f5ed67f6ca P 589db803901b4d34aac64364c0164d6c: stopping tablet replica
I20260806 22:50:20.026624 5637 raft_consensus.cc:2229] T 8b7b7bdfb66e438f9225f9f5ed67f6ca P 589db803901b4d34aac64364c0164d6c [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:50:20.026667 5636 ts_tablet_manager.cc:1905] T 8b7b7bdfb66e438f9225f9f5ed67f6ca P 3b7f0599c31b4cd798417dc934125fd0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:50:20.026710 5637 raft_consensus.cc:2258] T 8b7b7bdfb66e438f9225f9f5ed67f6ca P 589db803901b4d34aac64364c0164d6c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:50:20.026891 5635 ts_tablet_manager.cc:1905] T 8b7b7bdfb66e438f9225f9f5ed67f6ca P e2abad33dd7046b2bb589197ed70fa25: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:50:20.028525 5636 ts_tablet_manager.cc:1918] T 8b7b7bdfb66e438f9225f9f5ed67f6ca P 3b7f0599c31b4cd798417dc934125fd0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 22:50:20.028540 5637 ts_tablet_manager.cc:1905] T 8b7b7bdfb66e438f9225f9f5ed67f6ca P 589db803901b4d34aac64364c0164d6c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:50:20.028597 5636 log.cc:1196] T 8b7b7bdfb66e438f9225f9f5ed67f6ca P 3b7f0599c31b4cd798417dc934125fd0: Deleting WAL directory at /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056586788339-29547-0/minicluster-data/ts-2/wal/wals/8b7b7bdfb66e438f9225f9f5ed67f6ca
I20260806 22:50:20.028625 5635 ts_tablet_manager.cc:1918] T 8b7b7bdfb66e438f9225f9f5ed67f6ca P e2abad33dd7046b2bb589197ed70fa25: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 22:50:20.028678 5635 log.cc:1196] T 8b7b7bdfb66e438f9225f9f5ed67f6ca P e2abad33dd7046b2bb589197ed70fa25: Deleting WAL directory at /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056586788339-29547-0/minicluster-data/ts-1/wal/wals/8b7b7bdfb66e438f9225f9f5ed67f6ca
I20260806 22:50:20.028842 5636 ts_tablet_manager.cc:1939] T 8b7b7bdfb66e438f9225f9f5ed67f6ca P 3b7f0599c31b4cd798417dc934125fd0: Deleting consensus metadata
I20260806 22:50:20.028911 5635 ts_tablet_manager.cc:1939] T 8b7b7bdfb66e438f9225f9f5ed67f6ca P e2abad33dd7046b2bb589197ed70fa25: Deleting consensus metadata
I20260806 22:50:20.029187 5047 catalog_manager.cc:4841] TS e2abad33dd7046b2bb589197ed70fa25 (127.28.218.194:41871): tablet 8b7b7bdfb66e438f9225f9f5ed67f6ca (table test-table [id=00f76a94f6fa4bcaae86038662d11925]) successfully deleted
I20260806 22:50:20.029258 5047 catalog_manager.cc:4841] TS 3b7f0599c31b4cd798417dc934125fd0 (127.28.218.195:39813): tablet 8b7b7bdfb66e438f9225f9f5ed67f6ca (table test-table [id=00f76a94f6fa4bcaae86038662d11925]) successfully deleted
I20260806 22:50:20.029943 5637 ts_tablet_manager.cc:1918] T 8b7b7bdfb66e438f9225f9f5ed67f6ca P 589db803901b4d34aac64364c0164d6c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 22:50:20.030017 5637 log.cc:1196] T 8b7b7bdfb66e438f9225f9f5ed67f6ca P 589db803901b4d34aac64364c0164d6c: Deleting WAL directory at /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056586788339-29547-0/minicluster-data/ts-0/wal/wals/8b7b7bdfb66e438f9225f9f5ed67f6ca
I20260806 22:50:20.030232 5637 ts_tablet_manager.cc:1939] T 8b7b7bdfb66e438f9225f9f5ed67f6ca P 589db803901b4d34aac64364c0164d6c: Deleting consensus metadata
I20260806 22:50:20.031803 5046 catalog_manager.cc:4841] TS 589db803901b4d34aac64364c0164d6c (127.28.218.193:42059): tablet 8b7b7bdfb66e438f9225f9f5ed67f6ca (table test-table [id=00f76a94f6fa4bcaae86038662d11925]) successfully deleted
Aug 06 22:50:20 dist-test-slave-1xrh krb5kdc[5015](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056620, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 22:50:20 dist-test-slave-1xrh krb5kdc[5015](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.28.218.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:50:20 dist-test-slave-1xrh krb5kdc[5015](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.28.218.254@KRBTEST.COM, Server not found in Kerberos database
W20260806 22:50:20.054939 5123 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:50:20.049366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40690 (local address 127.28.218.254:33353)
0806 22:50:20.049442 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:20.049445 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:20.049582 (+ 137us) server_negotiation.cc:407] Connection header received
0806 22:50:20.049666 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:20.049667 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:20.049678 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:20.049749 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:20.050314 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:20.050815 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:20.051653 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:20.051843 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:20.051926 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:20.054874 (+ 2948us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40690: BlockingRecv error: recv got EOF from 127.0.0.1:40690 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 06 22:50:20 dist-test-slave-1xrh krb5kdc[5015](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056620, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.28.218.254@KRBTEST.COM
I20260806 22:50:20.062856 5123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:20.055760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40694 (local address 127.28.218.254:33353)
0806 22:50:20.055809 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:20.055811 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:20.055835 (+ 24us) server_negotiation.cc:407] Connection header received
0806 22:50:20.055904 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:20.055904 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:20.055917 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:20.055964 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:20.056591 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:20.057029 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:20.057972 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:20.058141 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:20.058193 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:20.060316 (+ 2123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:50:20.060318 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:50:20.060318 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:50:20.060325 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:50:20.061714 (+ 1389us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:20.062064 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:20.062065 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:20.062066 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:20.062126 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:20.062330 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:20.062330 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:20.062331 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:20.062439 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:50:20.062555 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:20.062674 (+ 119us) server_negotiation.cc:299] Negotiation successful
0806 22:50:20.062747 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260806 22:50:20.064384 5060 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40694:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:50:20.064531 5060 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:50:20.065929 5496 tablet_service.cc:1467] Processing CreateTablet for tablet 28e5a69786534d5ab3fe5b3574c823f3 (DEFAULT_TABLE table=test-table [id=9f06a09f97fe465399e5b70557c0b3b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:50:20.065941 5217 tablet_service.cc:1467] Processing CreateTablet for tablet 28e5a69786534d5ab3fe5b3574c823f3 (DEFAULT_TABLE table=test-table [id=9f06a09f97fe465399e5b70557c0b3b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:50:20.065976 5366 tablet_service.cc:1467] Processing CreateTablet for tablet 28e5a69786534d5ab3fe5b3574c823f3 (DEFAULT_TABLE table=test-table [id=9f06a09f97fe465399e5b70557c0b3b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:50:20.066074 5496 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 28e5a69786534d5ab3fe5b3574c823f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:50:20.066078 5217 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 28e5a69786534d5ab3fe5b3574c823f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:50:20.066082 5366 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 28e5a69786534d5ab3fe5b3574c823f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:50:20.067423 5613 tablet_bootstrap.cc:492] T 28e5a69786534d5ab3fe5b3574c823f3 P 3b7f0599c31b4cd798417dc934125fd0: Bootstrap starting.
I20260806 22:50:20.067428 5612 tablet_bootstrap.cc:492] T 28e5a69786534d5ab3fe5b3574c823f3 P e2abad33dd7046b2bb589197ed70fa25: Bootstrap starting.
I20260806 22:50:20.067803 5612 tablet_bootstrap.cc:654] T 28e5a69786534d5ab3fe5b3574c823f3 P e2abad33dd7046b2bb589197ed70fa25: Neither blocks nor log segments found. Creating new log.
I20260806 22:50:20.067803 5613 tablet_bootstrap.cc:654] T 28e5a69786534d5ab3fe5b3574c823f3 P 3b7f0599c31b4cd798417dc934125fd0: Neither blocks nor log segments found. Creating new log.
I20260806 22:50:20.068116 5618 tablet_bootstrap.cc:492] T 28e5a69786534d5ab3fe5b3574c823f3 P 589db803901b4d34aac64364c0164d6c: Bootstrap starting.
I20260806 22:50:20.068464 5613 tablet_bootstrap.cc:492] T 28e5a69786534d5ab3fe5b3574c823f3 P 3b7f0599c31b4cd798417dc934125fd0: No bootstrap required, opened a new log
I20260806 22:50:20.068513 5613 ts_tablet_manager.cc:1397] T 28e5a69786534d5ab3fe5b3574c823f3 P 3b7f0599c31b4cd798417dc934125fd0: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 22:50:20.068570 5618 tablet_bootstrap.cc:654] T 28e5a69786534d5ab3fe5b3574c823f3 P 589db803901b4d34aac64364c0164d6c: Neither blocks nor log segments found. Creating new log.
I20260806 22:50:20.068616 5613 raft_consensus.cc:348] T 28e5a69786534d5ab3fe5b3574c823f3 P 3b7f0599c31b4cd798417dc934125fd0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "589db803901b4d34aac64364c0164d6c" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 42059 } } peers { permanent_uuid: "e2abad33dd7046b2bb589197ed70fa25" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 41871 } } peers { permanent_uuid: "3b7f0599c31b4cd798417dc934125fd0" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 39813 } }
I20260806 22:50:20.068677 5613 raft_consensus.cc:374] T 28e5a69786534d5ab3fe5b3574c823f3 P 3b7f0599c31b4cd798417dc934125fd0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:50:20.068697 5613 raft_consensus.cc:729] T 28e5a69786534d5ab3fe5b3574c823f3 P 3b7f0599c31b4cd798417dc934125fd0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b7f0599c31b4cd798417dc934125fd0, State: Initialized, Role: FOLLOWER
I20260806 22:50:20.068727 5613 consensus_queue.cc:260] T 28e5a69786534d5ab3fe5b3574c823f3 P 3b7f0599c31b4cd798417dc934125fd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "589db803901b4d34aac64364c0164d6c" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 42059 } } peers { permanent_uuid: "e2abad33dd7046b2bb589197ed70fa25" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 41871 } } peers { permanent_uuid: "3b7f0599c31b4cd798417dc934125fd0" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 39813 } }
I20260806 22:50:20.068805 5613 ts_tablet_manager.cc:1428] T 28e5a69786534d5ab3fe5b3574c823f3 P 3b7f0599c31b4cd798417dc934125fd0: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:20.069168 5618 tablet_bootstrap.cc:492] T 28e5a69786534d5ab3fe5b3574c823f3 P 589db803901b4d34aac64364c0164d6c: No bootstrap required, opened a new log
I20260806 22:50:20.069219 5618 ts_tablet_manager.cc:1397] T 28e5a69786534d5ab3fe5b3574c823f3 P 589db803901b4d34aac64364c0164d6c: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 22:50:20.069312 5612 tablet_bootstrap.cc:492] T 28e5a69786534d5ab3fe5b3574c823f3 P e2abad33dd7046b2bb589197ed70fa25: No bootstrap required, opened a new log
I20260806 22:50:20.069358 5612 ts_tablet_manager.cc:1397] T 28e5a69786534d5ab3fe5b3574c823f3 P e2abad33dd7046b2bb589197ed70fa25: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260806 22:50:20.069420 5618 raft_consensus.cc:348] T 28e5a69786534d5ab3fe5b3574c823f3 P 589db803901b4d34aac64364c0164d6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "589db803901b4d34aac64364c0164d6c" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 42059 } } peers { permanent_uuid: "e2abad33dd7046b2bb589197ed70fa25" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 41871 } } peers { permanent_uuid: "3b7f0599c31b4cd798417dc934125fd0" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 39813 } }
I20260806 22:50:20.069492 5618 raft_consensus.cc:374] T 28e5a69786534d5ab3fe5b3574c823f3 P 589db803901b4d34aac64364c0164d6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:50:20.069515 5618 raft_consensus.cc:729] T 28e5a69786534d5ab3fe5b3574c823f3 P 589db803901b4d34aac64364c0164d6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 589db803901b4d34aac64364c0164d6c, State: Initialized, Role: FOLLOWER
I20260806 22:50:20.069492 5612 raft_consensus.cc:348] T 28e5a69786534d5ab3fe5b3574c823f3 P e2abad33dd7046b2bb589197ed70fa25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "589db803901b4d34aac64364c0164d6c" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 42059 } } peers { permanent_uuid: "e2abad33dd7046b2bb589197ed70fa25" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 41871 } } peers { permanent_uuid: "3b7f0599c31b4cd798417dc934125fd0" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 39813 } }
I20260806 22:50:20.069563 5612 raft_consensus.cc:374] T 28e5a69786534d5ab3fe5b3574c823f3 P e2abad33dd7046b2bb589197ed70fa25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:50:20.069554 5618 consensus_queue.cc:260] T 28e5a69786534d5ab3fe5b3574c823f3 P 589db803901b4d34aac64364c0164d6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "589db803901b4d34aac64364c0164d6c" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 42059 } } peers { permanent_uuid: "e2abad33dd7046b2bb589197ed70fa25" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 41871 } } peers { permanent_uuid: "3b7f0599c31b4cd798417dc934125fd0" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 39813 } }
I20260806 22:50:20.069590 5612 raft_consensus.cc:729] T 28e5a69786534d5ab3fe5b3574c823f3 P e2abad33dd7046b2bb589197ed70fa25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2abad33dd7046b2bb589197ed70fa25, State: Initialized, Role: FOLLOWER
I20260806 22:50:20.069635 5612 consensus_queue.cc:260] T 28e5a69786534d5ab3fe5b3574c823f3 P e2abad33dd7046b2bb589197ed70fa25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "589db803901b4d34aac64364c0164d6c" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 42059 } } peers { permanent_uuid: "e2abad33dd7046b2bb589197ed70fa25" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 41871 } } peers { permanent_uuid: "3b7f0599c31b4cd798417dc934125fd0" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 39813 } }
I20260806 22:50:20.069671 5618 ts_tablet_manager.cc:1428] T 28e5a69786534d5ab3fe5b3574c823f3 P 589db803901b4d34aac64364c0164d6c: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260806 22:50:20.069732 5612 ts_tablet_manager.cc:1428] T 28e5a69786534d5ab3fe5b3574c823f3 P e2abad33dd7046b2bb589197ed70fa25: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:20.103245 5670 raft_consensus.cc:482] T 28e5a69786534d5ab3fe5b3574c823f3 P 589db803901b4d34aac64364c0164d6c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:50:20.103358 5670 raft_consensus.cc:504] T 28e5a69786534d5ab3fe5b3574c823f3 P 589db803901b4d34aac64364c0164d6c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "589db803901b4d34aac64364c0164d6c" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 42059 } } peers { permanent_uuid: "e2abad33dd7046b2bb589197ed70fa25" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 41871 } } peers { permanent_uuid: "3b7f0599c31b4cd798417dc934125fd0" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 39813 } }
I20260806 22:50:20.103528 5670 leader_election.cc:288] T 28e5a69786534d5ab3fe5b3574c823f3 P 589db803901b4d34aac64364c0164d6c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e2abad33dd7046b2bb589197ed70fa25 (127.28.218.194:41871), 3b7f0599c31b4cd798417dc934125fd0 (127.28.218.195:39813)
I20260806 22:50:20.103730 5389 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28e5a69786534d5ab3fe5b3574c823f3" candidate_uuid: "589db803901b4d34aac64364c0164d6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2abad33dd7046b2bb589197ed70fa25" is_pre_election: true
I20260806 22:50:20.103869 5389 raft_consensus.cc:2454] T 28e5a69786534d5ab3fe5b3574c823f3 P e2abad33dd7046b2bb589197ed70fa25 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 589db803901b4d34aac64364c0164d6c in term 0.
I20260806 22:50:20.103868 5530 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28e5a69786534d5ab3fe5b3574c823f3" candidate_uuid: "589db803901b4d34aac64364c0164d6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b7f0599c31b4cd798417dc934125fd0" is_pre_election: true
I20260806 22:50:20.103983 5530 raft_consensus.cc:2454] T 28e5a69786534d5ab3fe5b3574c823f3 P 3b7f0599c31b4cd798417dc934125fd0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 589db803901b4d34aac64364c0164d6c in term 0.
I20260806 22:50:20.104034 5168 leader_election.cc:302] T 28e5a69786534d5ab3fe5b3574c823f3 P 589db803901b4d34aac64364c0164d6c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 589db803901b4d34aac64364c0164d6c, e2abad33dd7046b2bb589197ed70fa25; no voters:
I20260806 22:50:20.104143 5670 raft_consensus.cc:2781] T 28e5a69786534d5ab3fe5b3574c823f3 P 589db803901b4d34aac64364c0164d6c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:50:20.104166 5670 raft_consensus.cc:482] T 28e5a69786534d5ab3fe5b3574c823f3 P 589db803901b4d34aac64364c0164d6c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:50:20.104184 5670 raft_consensus.cc:3037] T 28e5a69786534d5ab3fe5b3574c823f3 P 589db803901b4d34aac64364c0164d6c [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:50:20.104931 5670 raft_consensus.cc:504] T 28e5a69786534d5ab3fe5b3574c823f3 P 589db803901b4d34aac64364c0164d6c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "589db803901b4d34aac64364c0164d6c" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 42059 } } peers { permanent_uuid: "e2abad33dd7046b2bb589197ed70fa25" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 41871 } } peers { permanent_uuid: "3b7f0599c31b4cd798417dc934125fd0" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 39813 } }
I20260806 22:50:20.105083 5670 leader_election.cc:288] T 28e5a69786534d5ab3fe5b3574c823f3 P 589db803901b4d34aac64364c0164d6c [CANDIDATE]: Term 1 election: Requested vote from peers e2abad33dd7046b2bb589197ed70fa25 (127.28.218.194:41871), 3b7f0599c31b4cd798417dc934125fd0 (127.28.218.195:39813)
I20260806 22:50:20.105286 5530 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28e5a69786534d5ab3fe5b3574c823f3" candidate_uuid: "589db803901b4d34aac64364c0164d6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b7f0599c31b4cd798417dc934125fd0"
I20260806 22:50:20.105285 5389 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28e5a69786534d5ab3fe5b3574c823f3" candidate_uuid: "589db803901b4d34aac64364c0164d6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2abad33dd7046b2bb589197ed70fa25"
I20260806 22:50:20.105351 5530 raft_consensus.cc:3037] T 28e5a69786534d5ab3fe5b3574c823f3 P 3b7f0599c31b4cd798417dc934125fd0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:50:20.105351 5389 raft_consensus.cc:3037] T 28e5a69786534d5ab3fe5b3574c823f3 P e2abad33dd7046b2bb589197ed70fa25 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:50:20.106047 5389 raft_consensus.cc:2454] T 28e5a69786534d5ab3fe5b3574c823f3 P e2abad33dd7046b2bb589197ed70fa25 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 589db803901b4d34aac64364c0164d6c in term 1.
I20260806 22:50:20.106212 5530 raft_consensus.cc:2454] T 28e5a69786534d5ab3fe5b3574c823f3 P 3b7f0599c31b4cd798417dc934125fd0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 589db803901b4d34aac64364c0164d6c in term 1.
I20260806 22:50:20.106243 5168 leader_election.cc:302] T 28e5a69786534d5ab3fe5b3574c823f3 P 589db803901b4d34aac64364c0164d6c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 589db803901b4d34aac64364c0164d6c, e2abad33dd7046b2bb589197ed70fa25; no voters:
I20260806 22:50:20.106547 5670 raft_consensus.cc:2781] T 28e5a69786534d5ab3fe5b3574c823f3 P 589db803901b4d34aac64364c0164d6c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:50:20.106616 5670 raft_consensus.cc:686] T 28e5a69786534d5ab3fe5b3574c823f3 P 589db803901b4d34aac64364c0164d6c [term 1 LEADER]: Becoming Leader. State: Replica: 589db803901b4d34aac64364c0164d6c, State: Running, Role: LEADER
I20260806 22:50:20.106670 5670 consensus_queue.cc:237] T 28e5a69786534d5ab3fe5b3574c823f3 P 589db803901b4d34aac64364c0164d6c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "589db803901b4d34aac64364c0164d6c" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 42059 } } peers { permanent_uuid: "e2abad33dd7046b2bb589197ed70fa25" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 41871 } } peers { permanent_uuid: "3b7f0599c31b4cd798417dc934125fd0" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 39813 } }
I20260806 22:50:20.107232 5061 catalog_manager.cc:5495] T 28e5a69786534d5ab3fe5b3574c823f3 P 589db803901b4d34aac64364c0164d6c reported cstate change: term changed from 0 to 1, leader changed from <none> to 589db803901b4d34aac64364c0164d6c (127.28.218.193). New cstate: current_term: 1 leader_uuid: "589db803901b4d34aac64364c0164d6c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "589db803901b4d34aac64364c0164d6c" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 42059 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e2abad33dd7046b2bb589197ed70fa25" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 41871 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b7f0599c31b4cd798417dc934125fd0" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 39813 } health_report { overall_health: UNKNOWN } } }
I20260806 22:50:20.116931 5609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:20.114548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44916 (local address 127.28.218.193:42059)
0806 22:50:20.114590 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:20.114592 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:20.114601 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:50:20.114622 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:20.114623 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:20.114633 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:20.114706 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:50:20.114861 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:20.115548 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:20.116225 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:20.116529 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:20.116603 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:20.116721 (+ 118us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:50:20.116753 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:50:20.116803 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:20.116846 (+ 43us) server_negotiation.cc:299] Negotiation successful
0806 22:50:20.116867 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 22:50:20.117553 5530 raft_consensus.cc:1261] T 28e5a69786534d5ab3fe5b3574c823f3 P 3b7f0599c31b4cd798417dc934125fd0 [term 1 FOLLOWER]: Refusing update from remote peer 589db803901b4d34aac64364c0164d6c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:50:20.117561 5389 raft_consensus.cc:1261] T 28e5a69786534d5ab3fe5b3574c823f3 P e2abad33dd7046b2bb589197ed70fa25 [term 1 FOLLOWER]: Refusing update from remote peer 589db803901b4d34aac64364c0164d6c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:50:20.117769 5665 consensus_queue.cc:1035] T 28e5a69786534d5ab3fe5b3574c823f3 P 589db803901b4d34aac64364c0164d6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2abad33dd7046b2bb589197ed70fa25" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 41871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:50:20.117848 5670 consensus_queue.cc:1035] T 28e5a69786534d5ab3fe5b3574c823f3 P 589db803901b4d34aac64364c0164d6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b7f0599c31b4cd798417dc934125fd0" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 39813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:50:20.120321 5061 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40694:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:50:20.120401 5061 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40694:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:50:20.121026 5061 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ecdca47ce9f544afa6f7395b0ad3445e: Sending DeleteTablet for 3 replicas of tablet 28e5a69786534d5ab3fe5b3574c823f3
I20260806 22:50:20.121248 5217 tablet_service.cc:1514] Processing DeleteTablet for tablet 28e5a69786534d5ab3fe5b3574c823f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:50:20 UTC) from {username='oryx', principal='oryx/127.28.218.254@KRBTEST.COM'} at 127.0.0.1:44890
I20260806 22:50:20.121282 5216 tablet_service.cc:1514] Processing DeleteTablet for tablet 28e5a69786534d5ab3fe5b3574c823f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:50:20 UTC) from {username='oryx', principal='oryx/127.28.218.254@KRBTEST.COM'} at 127.0.0.1:44890
I20260806 22:50:20.121282 5366 tablet_service.cc:1514] Processing DeleteTablet for tablet 28e5a69786534d5ab3fe5b3574c823f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:50:20 UTC) from {username='oryx', principal='oryx/127.28.218.254@KRBTEST.COM'} at 127.0.0.1:47640
I20260806 22:50:20.121359 5635 tablet_replica.cc:330] T 28e5a69786534d5ab3fe5b3574c823f3 P e2abad33dd7046b2bb589197ed70fa25: stopping tablet replica
I20260806 22:50:20.121357 5637 tablet_replica.cc:330] T 28e5a69786534d5ab3fe5b3574c823f3 P 589db803901b4d34aac64364c0164d6c: stopping tablet replica
I20260806 22:50:20.121403 5637 raft_consensus.cc:2229] T 28e5a69786534d5ab3fe5b3574c823f3 P 589db803901b4d34aac64364c0164d6c [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:50:20.121417 5635 raft_consensus.cc:2229] T 28e5a69786534d5ab3fe5b3574c823f3 P e2abad33dd7046b2bb589197ed70fa25 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:50:20.121454 5635 raft_consensus.cc:2258] T 28e5a69786534d5ab3fe5b3574c823f3 P e2abad33dd7046b2bb589197ed70fa25 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:50:20.121467 5637 raft_consensus.cc:2258] T 28e5a69786534d5ab3fe5b3574c823f3 P 589db803901b4d34aac64364c0164d6c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:50:20.121588 5496 tablet_service.cc:1514] Processing DeleteTablet for tablet 28e5a69786534d5ab3fe5b3574c823f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:50:20 UTC) from {username='oryx', principal='oryx/127.28.218.254@KRBTEST.COM'} at 127.0.0.1:33464
I20260806 22:50:20.121671 5636 tablet_replica.cc:330] T 28e5a69786534d5ab3fe5b3574c823f3 P 3b7f0599c31b4cd798417dc934125fd0: stopping tablet replica
I20260806 22:50:20.121716 5636 raft_consensus.cc:2229] T 28e5a69786534d5ab3fe5b3574c823f3 P 3b7f0599c31b4cd798417dc934125fd0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:50:20.121776 5636 raft_consensus.cc:2258] T 28e5a69786534d5ab3fe5b3574c823f3 P 3b7f0599c31b4cd798417dc934125fd0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:50:20.121865 5061 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40694:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:50:20.122006 5061 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:50:20.122182 5636 ts_tablet_manager.cc:1905] T 28e5a69786534d5ab3fe5b3574c823f3 P 3b7f0599c31b4cd798417dc934125fd0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:50:20.123574 5635 ts_tablet_manager.cc:1905] T 28e5a69786534d5ab3fe5b3574c823f3 P e2abad33dd7046b2bb589197ed70fa25: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:50:20.123993 5637 ts_tablet_manager.cc:1905] T 28e5a69786534d5ab3fe5b3574c823f3 P 589db803901b4d34aac64364c0164d6c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:50:20.124956 5635 ts_tablet_manager.cc:1918] T 28e5a69786534d5ab3fe5b3574c823f3 P e2abad33dd7046b2bb589197ed70fa25: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:50:20.124995 5637 ts_tablet_manager.cc:1918] T 28e5a69786534d5ab3fe5b3574c823f3 P 589db803901b4d34aac64364c0164d6c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:50:20.125006 5635 log.cc:1196] T 28e5a69786534d5ab3fe5b3574c823f3 P e2abad33dd7046b2bb589197ed70fa25: Deleting WAL directory at /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056586788339-29547-0/minicluster-data/ts-1/wal/wals/28e5a69786534d5ab3fe5b3574c823f3
I20260806 22:50:20.125051 5637 log.cc:1196] T 28e5a69786534d5ab3fe5b3574c823f3 P 589db803901b4d34aac64364c0164d6c: Deleting WAL directory at /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056586788339-29547-0/minicluster-data/ts-0/wal/wals/28e5a69786534d5ab3fe5b3574c823f3
I20260806 22:50:20.125239 5635 ts_tablet_manager.cc:1939] T 28e5a69786534d5ab3fe5b3574c823f3 P e2abad33dd7046b2bb589197ed70fa25: Deleting consensus metadata
I20260806 22:50:20.125303 5637 ts_tablet_manager.cc:1939] T 28e5a69786534d5ab3fe5b3574c823f3 P 589db803901b4d34aac64364c0164d6c: Deleting consensus metadata
I20260806 22:50:20.125440 5366 tablet_service.cc:1467] Processing CreateTablet for tablet 3060a181823d4b8a8860e9ed8c67ccee (DEFAULT_TABLE table=test-table [id=4cff1aa6a1b14deba94a1fe0485cdddf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:50:20.125556 5366 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3060a181823d4b8a8860e9ed8c67ccee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:50:20.126396 5612 tablet_bootstrap.cc:492] T 3060a181823d4b8a8860e9ed8c67ccee P e2abad33dd7046b2bb589197ed70fa25: Bootstrap starting.
I20260806 22:50:20.126628 5496 tablet_service.cc:1467] Processing CreateTablet for tablet 3060a181823d4b8a8860e9ed8c67ccee (DEFAULT_TABLE table=test-table [id=4cff1aa6a1b14deba94a1fe0485cdddf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:50:20.126921 5046 catalog_manager.cc:4841] TS 589db803901b4d34aac64364c0164d6c (127.28.218.193:42059): tablet 28e5a69786534d5ab3fe5b3574c823f3 (table test-table [id=9f06a09f97fe465399e5b70557c0b3b3]) successfully deleted
W20260806 22:50:20.127044 5046 catalog_manager.cc:4808] TS 589db803901b4d34aac64364c0164d6c (127.28.218.193:42059): delete failed for tablet 28e5a69786534d5ab3fe5b3574c823f3 because the tablet was not found. No further retry: Not found: Tablet not found: 28e5a69786534d5ab3fe5b3574c823f3
I20260806 22:50:20.127079 5612 tablet_bootstrap.cc:654] T 3060a181823d4b8a8860e9ed8c67ccee P e2abad33dd7046b2bb589197ed70fa25: Neither blocks nor log segments found. Creating new log.
I20260806 22:50:20.127281 5216 tablet_service.cc:1467] Processing CreateTablet for tablet 3060a181823d4b8a8860e9ed8c67ccee (DEFAULT_TABLE table=test-table [id=4cff1aa6a1b14deba94a1fe0485cdddf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:50:20.127390 5216 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3060a181823d4b8a8860e9ed8c67ccee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:50:20.128281 5618 tablet_bootstrap.cc:492] T 3060a181823d4b8a8860e9ed8c67ccee P 589db803901b4d34aac64364c0164d6c: Bootstrap starting.
I20260806 22:50:20.128546 5636 ts_tablet_manager.cc:1918] T 28e5a69786534d5ab3fe5b3574c823f3 P 3b7f0599c31b4cd798417dc934125fd0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:50:20.128603 5618 tablet_bootstrap.cc:654] T 3060a181823d4b8a8860e9ed8c67ccee P 589db803901b4d34aac64364c0164d6c: Neither blocks nor log segments found. Creating new log.
I20260806 22:50:20.128611 5636 log.cc:1196] T 28e5a69786534d5ab3fe5b3574c823f3 P 3b7f0599c31b4cd798417dc934125fd0: Deleting WAL directory at /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056586788339-29547-0/minicluster-data/ts-2/wal/wals/28e5a69786534d5ab3fe5b3574c823f3
I20260806 22:50:20.128839 5636 ts_tablet_manager.cc:1939] T 28e5a69786534d5ab3fe5b3574c823f3 P 3b7f0599c31b4cd798417dc934125fd0: Deleting consensus metadata
I20260806 22:50:20.128999 5496 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3060a181823d4b8a8860e9ed8c67ccee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:50:20.129110 5618 tablet_bootstrap.cc:492] T 3060a181823d4b8a8860e9ed8c67ccee P 589db803901b4d34aac64364c0164d6c: No bootstrap required, opened a new log
I20260806 22:50:20.129150 5618 ts_tablet_manager.cc:1397] T 3060a181823d4b8a8860e9ed8c67ccee P 589db803901b4d34aac64364c0164d6c: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260806 22:50:20.129204 5047 catalog_manager.cc:4841] TS 3b7f0599c31b4cd798417dc934125fd0 (127.28.218.195:39813): tablet 28e5a69786534d5ab3fe5b3574c823f3 (table test-table [id=9f06a09f97fe465399e5b70557c0b3b3]) successfully deleted
I20260806 22:50:20.129240 5618 raft_consensus.cc:348] T 3060a181823d4b8a8860e9ed8c67ccee P 589db803901b4d34aac64364c0164d6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "589db803901b4d34aac64364c0164d6c" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 42059 } } peers { permanent_uuid: "e2abad33dd7046b2bb589197ed70fa25" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 41871 } } peers { permanent_uuid: "3b7f0599c31b4cd798417dc934125fd0" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 39813 } }
I20260806 22:50:20.129292 5618 raft_consensus.cc:374] T 3060a181823d4b8a8860e9ed8c67ccee P 589db803901b4d34aac64364c0164d6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:50:20.129311 5618 raft_consensus.cc:729] T 3060a181823d4b8a8860e9ed8c67ccee P 589db803901b4d34aac64364c0164d6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 589db803901b4d34aac64364c0164d6c, State: Initialized, Role: FOLLOWER
I20260806 22:50:20.129350 5618 consensus_queue.cc:260] T 3060a181823d4b8a8860e9ed8c67ccee P 589db803901b4d34aac64364c0164d6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "589db803901b4d34aac64364c0164d6c" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 42059 } } peers { permanent_uuid: "e2abad33dd7046b2bb589197ed70fa25" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 41871 } } peers { permanent_uuid: "3b7f0599c31b4cd798417dc934125fd0" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 39813 } }
I20260806 22:50:20.129426 5618 ts_tablet_manager.cc:1428] T 3060a181823d4b8a8860e9ed8c67ccee P 589db803901b4d34aac64364c0164d6c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:20.130132 5613 tablet_bootstrap.cc:492] T 3060a181823d4b8a8860e9ed8c67ccee P 3b7f0599c31b4cd798417dc934125fd0: Bootstrap starting.
I20260806 22:50:20.130288 5047 catalog_manager.cc:4841] TS e2abad33dd7046b2bb589197ed70fa25 (127.28.218.194:41871): tablet 28e5a69786534d5ab3fe5b3574c823f3 (table test-table [id=9f06a09f97fe465399e5b70557c0b3b3]) successfully deleted
I20260806 22:50:20.130455 5613 tablet_bootstrap.cc:654] T 3060a181823d4b8a8860e9ed8c67ccee P 3b7f0599c31b4cd798417dc934125fd0: Neither blocks nor log segments found. Creating new log.
I20260806 22:50:20.130638 5612 tablet_bootstrap.cc:492] T 3060a181823d4b8a8860e9ed8c67ccee P e2abad33dd7046b2bb589197ed70fa25: No bootstrap required, opened a new log
I20260806 22:50:20.130682 5612 ts_tablet_manager.cc:1397] T 3060a181823d4b8a8860e9ed8c67ccee P e2abad33dd7046b2bb589197ed70fa25: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20260806 22:50:20.130826 5612 raft_consensus.cc:348] T 3060a181823d4b8a8860e9ed8c67ccee P e2abad33dd7046b2bb589197ed70fa25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "589db803901b4d34aac64364c0164d6c" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 42059 } } peers { permanent_uuid: "e2abad33dd7046b2bb589197ed70fa25" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 41871 } } peers { permanent_uuid: "3b7f0599c31b4cd798417dc934125fd0" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 39813 } }
I20260806 22:50:20.130926 5612 raft_consensus.cc:374] T 3060a181823d4b8a8860e9ed8c67ccee P e2abad33dd7046b2bb589197ed70fa25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:50:20.130959 5612 raft_consensus.cc:729] T 3060a181823d4b8a8860e9ed8c67ccee P e2abad33dd7046b2bb589197ed70fa25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2abad33dd7046b2bb589197ed70fa25, State: Initialized, Role: FOLLOWER
I20260806 22:50:20.130996 5612 consensus_queue.cc:260] T 3060a181823d4b8a8860e9ed8c67ccee P e2abad33dd7046b2bb589197ed70fa25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "589db803901b4d34aac64364c0164d6c" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 42059 } } peers { permanent_uuid: "e2abad33dd7046b2bb589197ed70fa25" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 41871 } } peers { permanent_uuid: "3b7f0599c31b4cd798417dc934125fd0" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 39813 } }
I20260806 22:50:20.130998 5613 tablet_bootstrap.cc:492] T 3060a181823d4b8a8860e9ed8c67ccee P 3b7f0599c31b4cd798417dc934125fd0: No bootstrap required, opened a new log
I20260806 22:50:20.131054 5613 ts_tablet_manager.cc:1397] T 3060a181823d4b8a8860e9ed8c67ccee P 3b7f0599c31b4cd798417dc934125fd0: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 22:50:20.131088 5612 ts_tablet_manager.cc:1428] T 3060a181823d4b8a8860e9ed8c67ccee P e2abad33dd7046b2bb589197ed70fa25: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260806 22:50:20.131175 5613 raft_consensus.cc:348] T 3060a181823d4b8a8860e9ed8c67ccee P 3b7f0599c31b4cd798417dc934125fd0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "589db803901b4d34aac64364c0164d6c" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 42059 } } peers { permanent_uuid: "e2abad33dd7046b2bb589197ed70fa25" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 41871 } } peers { permanent_uuid: "3b7f0599c31b4cd798417dc934125fd0" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 39813 } }
I20260806 22:50:20.131244 5613 raft_consensus.cc:374] T 3060a181823d4b8a8860e9ed8c67ccee P 3b7f0599c31b4cd798417dc934125fd0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:50:20.131263 5613 raft_consensus.cc:729] T 3060a181823d4b8a8860e9ed8c67ccee P 3b7f0599c31b4cd798417dc934125fd0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b7f0599c31b4cd798417dc934125fd0, State: Initialized, Role: FOLLOWER
I20260806 22:50:20.131301 5613 consensus_queue.cc:260] T 3060a181823d4b8a8860e9ed8c67ccee P 3b7f0599c31b4cd798417dc934125fd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "589db803901b4d34aac64364c0164d6c" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 42059 } } peers { permanent_uuid: "e2abad33dd7046b2bb589197ed70fa25" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 41871 } } peers { permanent_uuid: "3b7f0599c31b4cd798417dc934125fd0" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 39813 } }
I20260806 22:50:20.131384 5613 ts_tablet_manager.cc:1428] T 3060a181823d4b8a8860e9ed8c67ccee P 3b7f0599c31b4cd798417dc934125fd0: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:20.240741 5617 raft_consensus.cc:482] T 3060a181823d4b8a8860e9ed8c67ccee P e2abad33dd7046b2bb589197ed70fa25 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:50:20.240854 5617 raft_consensus.cc:504] T 3060a181823d4b8a8860e9ed8c67ccee P e2abad33dd7046b2bb589197ed70fa25 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "589db803901b4d34aac64364c0164d6c" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 42059 } } peers { permanent_uuid: "e2abad33dd7046b2bb589197ed70fa25" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 41871 } } peers { permanent_uuid: "3b7f0599c31b4cd798417dc934125fd0" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 39813 } }
I20260806 22:50:20.241255 5617 leader_election.cc:288] T 3060a181823d4b8a8860e9ed8c67ccee P e2abad33dd7046b2bb589197ed70fa25 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 589db803901b4d34aac64364c0164d6c (127.28.218.193:42059), 3b7f0599c31b4cd798417dc934125fd0 (127.28.218.195:39813)
I20260806 22:50:20.245395 5702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:20.241514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.194:34557 (local address 127.28.218.195:39813)
0806 22:50:20.241621 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:20.241623 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:20.241652 (+ 29us) server_negotiation.cc:407] Connection header received
0806 22:50:20.241737 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:20.241738 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:20.241749 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:20.241826 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:50:20.242069 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:20.242790 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:20.244465 (+ 1675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:20.245041 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:20.245108 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:20.245116 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:20.245282 (+ 166us) server_negotiation.cc:299] Negotiation successful
0806 22:50:20.245331 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87,"thread_start_us":51,"threads_started":1}
I20260806 22:50:20.245388 5609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:20.241546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.194:50065 (local address 127.28.218.193:42059)
0806 22:50:20.241612 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:20.241614 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:20.241660 (+ 46us) server_negotiation.cc:407] Connection header received
0806 22:50:20.241737 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:20.241738 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:20.241749 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:20.241818 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:50:20.242019 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:20.242790 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:20.244465 (+ 1675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:20.245022 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:20.245104 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:20.245114 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:20.245282 (+ 168us) server_negotiation.cc:299] Negotiation successful
0806 22:50:20.245317 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260806 22:50:20.245436 5701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:20.241443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.195:39813 (local address 127.28.218.194:34557)
0806 22:50:20.241590 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:20.241601 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:20.241667 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:20.241844 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:20.241845 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:20.241849 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:50:20.241976 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:20.241977 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:20.242874 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:20.242875 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:20.244363 (+ 1488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:20.244366 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:20.245142 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:20.245143 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:20.245203 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:20.245207 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:50:20.245261 (+ 54us) client_negotiation.cc:240] Negotiation successful
0806 22:50:20.245298 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"mutex_wait_us":133,"thread_start_us":71,"threads_started":1}
I20260806 22:50:20.245437 5700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:20.241444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.193:42059 (local address 127.28.218.194:50065)
0806 22:50:20.241603 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:20.241613 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:20.241667 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:20.241844 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:20.241845 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:20.241848 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:50:20.241929 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:20.241931 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:20.242874 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:20.242875 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:20.244363 (+ 1488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:20.244366 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:20.245117 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:20.245118 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:20.245186 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:20.245191 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:50:20.245252 (+ 61us) client_negotiation.cc:240] Negotiation successful
0806 22:50:20.245298 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"mutex_wait_us":85,"thread_start_us":81,"threads_started":1}
I20260806 22:50:20.245796 5242 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3060a181823d4b8a8860e9ed8c67ccee" candidate_uuid: "e2abad33dd7046b2bb589197ed70fa25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "589db803901b4d34aac64364c0164d6c" is_pre_election: true
I20260806 22:50:20.245893 5242 raft_consensus.cc:2454] T 3060a181823d4b8a8860e9ed8c67ccee P 589db803901b4d34aac64364c0164d6c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2abad33dd7046b2bb589197ed70fa25 in term 0.
I20260806 22:50:20.246073 5317 leader_election.cc:302] T 3060a181823d4b8a8860e9ed8c67ccee P e2abad33dd7046b2bb589197ed70fa25 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 589db803901b4d34aac64364c0164d6c, e2abad33dd7046b2bb589197ed70fa25; no voters:
I20260806 22:50:20.246106 5530 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3060a181823d4b8a8860e9ed8c67ccee" candidate_uuid: "e2abad33dd7046b2bb589197ed70fa25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b7f0599c31b4cd798417dc934125fd0" is_pre_election: true
I20260806 22:50:20.246207 5530 raft_consensus.cc:2454] T 3060a181823d4b8a8860e9ed8c67ccee P 3b7f0599c31b4cd798417dc934125fd0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2abad33dd7046b2bb589197ed70fa25 in term 0.
I20260806 22:50:20.246218 5617 raft_consensus.cc:2781] T 3060a181823d4b8a8860e9ed8c67ccee P e2abad33dd7046b2bb589197ed70fa25 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:50:20.246251 5617 raft_consensus.cc:482] T 3060a181823d4b8a8860e9ed8c67ccee P e2abad33dd7046b2bb589197ed70fa25 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:50:20.246282 5617 raft_consensus.cc:3037] T 3060a181823d4b8a8860e9ed8c67ccee P e2abad33dd7046b2bb589197ed70fa25 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:50:20.247009 5617 raft_consensus.cc:504] T 3060a181823d4b8a8860e9ed8c67ccee P e2abad33dd7046b2bb589197ed70fa25 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "589db803901b4d34aac64364c0164d6c" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 42059 } } peers { permanent_uuid: "e2abad33dd7046b2bb589197ed70fa25" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 41871 } } peers { permanent_uuid: "3b7f0599c31b4cd798417dc934125fd0" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 39813 } }
I20260806 22:50:20.247182 5617 leader_election.cc:288] T 3060a181823d4b8a8860e9ed8c67ccee P e2abad33dd7046b2bb589197ed70fa25 [CANDIDATE]: Term 1 election: Requested vote from peers 589db803901b4d34aac64364c0164d6c (127.28.218.193:42059), 3b7f0599c31b4cd798417dc934125fd0 (127.28.218.195:39813)
I20260806 22:50:20.247308 5530 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3060a181823d4b8a8860e9ed8c67ccee" candidate_uuid: "e2abad33dd7046b2bb589197ed70fa25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b7f0599c31b4cd798417dc934125fd0"
I20260806 22:50:20.247328 5242 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3060a181823d4b8a8860e9ed8c67ccee" candidate_uuid: "e2abad33dd7046b2bb589197ed70fa25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "589db803901b4d34aac64364c0164d6c"
I20260806 22:50:20.247399 5530 raft_consensus.cc:3037] T 3060a181823d4b8a8860e9ed8c67ccee P 3b7f0599c31b4cd798417dc934125fd0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:50:20.247403 5242 raft_consensus.cc:3037] T 3060a181823d4b8a8860e9ed8c67ccee P 589db803901b4d34aac64364c0164d6c [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:50:20.247975 5242 raft_consensus.cc:2454] T 3060a181823d4b8a8860e9ed8c67ccee P 589db803901b4d34aac64364c0164d6c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2abad33dd7046b2bb589197ed70fa25 in term 1.
I20260806 22:50:20.248122 5530 raft_consensus.cc:2454] T 3060a181823d4b8a8860e9ed8c67ccee P 3b7f0599c31b4cd798417dc934125fd0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2abad33dd7046b2bb589197ed70fa25 in term 1.
I20260806 22:50:20.248144 5317 leader_election.cc:302] T 3060a181823d4b8a8860e9ed8c67ccee P e2abad33dd7046b2bb589197ed70fa25 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 589db803901b4d34aac64364c0164d6c, e2abad33dd7046b2bb589197ed70fa25; no voters:
I20260806 22:50:20.248234 5617 raft_consensus.cc:2781] T 3060a181823d4b8a8860e9ed8c67ccee P e2abad33dd7046b2bb589197ed70fa25 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:50:20.248344 5617 raft_consensus.cc:686] T 3060a181823d4b8a8860e9ed8c67ccee P e2abad33dd7046b2bb589197ed70fa25 [term 1 LEADER]: Becoming Leader. State: Replica: e2abad33dd7046b2bb589197ed70fa25, State: Running, Role: LEADER
I20260806 22:50:20.248421 5617 consensus_queue.cc:237] T 3060a181823d4b8a8860e9ed8c67ccee P e2abad33dd7046b2bb589197ed70fa25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "589db803901b4d34aac64364c0164d6c" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 42059 } } peers { permanent_uuid: "e2abad33dd7046b2bb589197ed70fa25" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 41871 } } peers { permanent_uuid: "3b7f0599c31b4cd798417dc934125fd0" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 39813 } }
I20260806 22:50:20.249039 5061 catalog_manager.cc:5495] T 3060a181823d4b8a8860e9ed8c67ccee P e2abad33dd7046b2bb589197ed70fa25 reported cstate change: term changed from 0 to 1, leader changed from <none> to e2abad33dd7046b2bb589197ed70fa25 (127.28.218.194). New cstate: current_term: 1 leader_uuid: "e2abad33dd7046b2bb589197ed70fa25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "589db803901b4d34aac64364c0164d6c" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 42059 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2abad33dd7046b2bb589197ed70fa25" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 41871 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b7f0599c31b4cd798417dc934125fd0" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 39813 } health_report { overall_health: UNKNOWN } } }
I20260806 22:50:20.277912 5705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:20.275334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47650 (local address 127.28.218.194:41871)
0806 22:50:20.275467 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:20.275469 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:20.275477 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:50:20.275515 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:20.275515 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:20.275530 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:20.275601 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:50:20.275720 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:20.276142 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:20.277014 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:20.277191 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:20.277244 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:20.277418 (+ 174us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:50:20.277511 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:50:20.277742 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:20.277794 (+ 52us) server_negotiation.cc:299] Negotiation successful
0806 22:50:20.277813 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":91,"threads_started":1}
I20260806 22:50:20.281898 5609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:20.280056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.194:47303 (local address 127.28.218.193:42059)
0806 22:50:20.280113 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:20.280118 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:20.280127 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:50:20.280145 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:20.280145 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:20.280153 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:20.280197 (+ 44us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:50:20.280297 (+ 100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:20.280706 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:20.281382 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:20.281549 (+ 167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:20.281595 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:20.281736 (+ 141us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:50:20.281757 (+ 21us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:50:20.281789 (+ 32us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:20.281840 (+ 51us) server_negotiation.cc:299] Negotiation successful
0806 22:50:20.281853 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 22:50:20.281935 5708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:20.279881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.193:42059 (local address 127.28.218.194:47303)
0806 22:50:20.280027 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:20.280037 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:20.280077 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:20.280209 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:20.280210 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:20.280211 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 22:50:20.280259 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:20.280260 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:20.280756 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:20.280756 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:20.281313 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:20.281315 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:20.281617 (+ 302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:20.281618 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:20.281668 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:20.281679 (+ 11us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 22:50:20.281804 (+ 125us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 22:50:20.281805 (+ 1us) client_negotiation.cc:769] Sending connection context
0806 22:50:20.281839 (+ 34us) client_negotiation.cc:240] Negotiation successful
0806 22:50:20.281867 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":61,"threads_started":1}
I20260806 22:50:20.283897 5530 raft_consensus.cc:1261] T 3060a181823d4b8a8860e9ed8c67ccee P 3b7f0599c31b4cd798417dc934125fd0 [term 1 FOLLOWER]: Refusing update from remote peer e2abad33dd7046b2bb589197ed70fa25: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:50:20.283952 5242 raft_consensus.cc:1261] T 3060a181823d4b8a8860e9ed8c67ccee P 589db803901b4d34aac64364c0164d6c [term 1 FOLLOWER]: Refusing update from remote peer e2abad33dd7046b2bb589197ed70fa25: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:50:20.284152 5703 consensus_queue.cc:1035] T 3060a181823d4b8a8860e9ed8c67ccee P e2abad33dd7046b2bb589197ed70fa25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b7f0599c31b4cd798417dc934125fd0" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 39813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:50:20.284255 5617 consensus_queue.cc:1035] T 3060a181823d4b8a8860e9ed8c67ccee P e2abad33dd7046b2bb589197ed70fa25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "589db803901b4d34aac64364c0164d6c" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 42059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:50:20.290503 5664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:20.288383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.194:41871 (local address 127.28.218.193:48879)
0806 22:50:20.288436 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:20.288445 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:20.288477 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:20.288688 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:20.288689 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:20.288689 (+ 0us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 22:50:20.288723 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:20.288724 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:20.289283 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:20.289284 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:20.289773 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:20.289775 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:20.290080 (+ 305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:20.290081 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:20.290148 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:20.290155 (+ 7us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 22:50:20.290318 (+ 163us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 22:50:20.290321 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:50:20.290383 (+ 62us) client_negotiation.cc:240] Negotiation successful
0806 22:50:20.290431 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260806 22:50:20.290498 5705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:20.288541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.193:48879 (local address 127.28.218.194:41871)
0806 22:50:20.288590 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:20.288591 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:20.288597 (+ 6us) server_negotiation.cc:407] Connection header received
0806 22:50:20.288614 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:20.288615 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:20.288622 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:20.288685 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:50:20.288788 (+ 103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:20.289217 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:20.289840 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:20.290023 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:20.290069 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:20.290233 (+ 164us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:50:20.290265 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:50:20.290313 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:20.290387 (+ 74us) server_negotiation.cc:299] Negotiation successful
0806 22:50:20.290404 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 22:50:20.297551 5060 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40694:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:50:20.297626 5060 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40694:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:50:20.298228 5060 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ecdca47ce9f544afa6f7395b0ad3445e: Sending DeleteTablet for 3 replicas of tablet 3060a181823d4b8a8860e9ed8c67ccee
I20260806 22:50:20.298514 5366 tablet_service.cc:1514] Processing DeleteTablet for tablet 3060a181823d4b8a8860e9ed8c67ccee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:50:20 UTC) from {username='oryx', principal='oryx/127.28.218.254@KRBTEST.COM'} at 127.0.0.1:47640
I20260806 22:50:20.298532 5496 tablet_service.cc:1514] Processing DeleteTablet for tablet 3060a181823d4b8a8860e9ed8c67ccee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:50:20 UTC) from {username='oryx', principal='oryx/127.28.218.254@KRBTEST.COM'} at 127.0.0.1:33464
I20260806 22:50:20.298619 5636 tablet_replica.cc:330] T 3060a181823d4b8a8860e9ed8c67ccee P 3b7f0599c31b4cd798417dc934125fd0: stopping tablet replica
I20260806 22:50:20.298621 5635 tablet_replica.cc:330] T 3060a181823d4b8a8860e9ed8c67ccee P e2abad33dd7046b2bb589197ed70fa25: stopping tablet replica
I20260806 22:50:20.298609 5216 tablet_service.cc:1514] Processing DeleteTablet for tablet 3060a181823d4b8a8860e9ed8c67ccee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:50:20 UTC) from {username='oryx', principal='oryx/127.28.218.254@KRBTEST.COM'} at 127.0.0.1:44890
I20260806 22:50:20.298663 5636 raft_consensus.cc:2229] T 3060a181823d4b8a8860e9ed8c67ccee P 3b7f0599c31b4cd798417dc934125fd0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:50:20.298669 5635 raft_consensus.cc:2229] T 3060a181823d4b8a8860e9ed8c67ccee P e2abad33dd7046b2bb589197ed70fa25 [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:50:20.298683 5637 tablet_replica.cc:330] T 3060a181823d4b8a8860e9ed8c67ccee P 589db803901b4d34aac64364c0164d6c: stopping tablet replica
I20260806 22:50:20.298695 5636 raft_consensus.cc:2258] T 3060a181823d4b8a8860e9ed8c67ccee P 3b7f0599c31b4cd798417dc934125fd0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:50:20.298736 5637 raft_consensus.cc:2229] T 3060a181823d4b8a8860e9ed8c67ccee P 589db803901b4d34aac64364c0164d6c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:50:20.298753 5635 raft_consensus.cc:2258] T 3060a181823d4b8a8860e9ed8c67ccee P e2abad33dd7046b2bb589197ed70fa25 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:50:20.298763 5637 raft_consensus.cc:2258] T 3060a181823d4b8a8860e9ed8c67ccee P 589db803901b4d34aac64364c0164d6c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:50:20.299113 5637 ts_tablet_manager.cc:1905] T 3060a181823d4b8a8860e9ed8c67ccee P 589db803901b4d34aac64364c0164d6c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:50:20.299149 5636 ts_tablet_manager.cc:1905] T 3060a181823d4b8a8860e9ed8c67ccee P 3b7f0599c31b4cd798417dc934125fd0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:50:20.299269 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 5147
I20260806 22:50:20.300688 5636 ts_tablet_manager.cc:1918] T 3060a181823d4b8a8860e9ed8c67ccee P 3b7f0599c31b4cd798417dc934125fd0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 22:50:20.300736 5636 log.cc:1196] T 3060a181823d4b8a8860e9ed8c67ccee P 3b7f0599c31b4cd798417dc934125fd0: Deleting WAL directory at /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056586788339-29547-0/minicluster-data/ts-2/wal/wals/3060a181823d4b8a8860e9ed8c67ccee
I20260806 22:50:20.300927 5636 ts_tablet_manager.cc:1939] T 3060a181823d4b8a8860e9ed8c67ccee P 3b7f0599c31b4cd798417dc934125fd0: Deleting consensus metadata
I20260806 22:50:20.301810 5047 catalog_manager.cc:4841] TS 3b7f0599c31b4cd798417dc934125fd0 (127.28.218.195:39813): tablet 3060a181823d4b8a8860e9ed8c67ccee (table test-table [id=4cff1aa6a1b14deba94a1fe0485cdddf]) successfully deleted
I20260806 22:50:20.303349 5635 ts_tablet_manager.cc:1905] T 3060a181823d4b8a8860e9ed8c67ccee P e2abad33dd7046b2bb589197ed70fa25: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:50:20.305179 5635 ts_tablet_manager.cc:1918] T 3060a181823d4b8a8860e9ed8c67ccee P e2abad33dd7046b2bb589197ed70fa25: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 22:50:20.305244 5635 log.cc:1196] T 3060a181823d4b8a8860e9ed8c67ccee P e2abad33dd7046b2bb589197ed70fa25: Deleting WAL directory at /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056586788339-29547-0/minicluster-data/ts-1/wal/wals/3060a181823d4b8a8860e9ed8c67ccee
I20260806 22:50:20.305457 5635 ts_tablet_manager.cc:1939] T 3060a181823d4b8a8860e9ed8c67ccee P e2abad33dd7046b2bb589197ed70fa25: Deleting consensus metadata
I20260806 22:50:20.305771 5047 catalog_manager.cc:4841] TS e2abad33dd7046b2bb589197ed70fa25 (127.28.218.194:41871): tablet 3060a181823d4b8a8860e9ed8c67ccee (table test-table [id=4cff1aa6a1b14deba94a1fe0485cdddf]) successfully deleted
W20260806 22:50:20.308915 5046 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.28.218.193:42059 (error 108)
I20260806 22:50:20.309201 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 5299
I20260806 22:50:20.311841 5714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:20.311648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.193:42059 (local address 127.0.0.1:44926)
0806 22:50:20.311750 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:20.311794 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.193:42059: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84,"thread_start_us":63,"threads_started":1}
W20260806 22:50:20.311980 5046 catalog_manager.cc:4584] TS 589db803901b4d34aac64364c0164d6c (127.28.218.193:42059): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3060a181823d4b8a8860e9ed8c67ccee: Network error: Client connection negotiation failed: client connection to 127.28.218.193:42059: connect: Connection refused (error 111)
I20260806 22:50:20.315564 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 5445
I20260806 22:50:20.321516 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 5028
2026-08-06T22:50:20Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (2725 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786056586788339-29547-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:50:20 dist-test-slave-1xrh krb5kdc[5717](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:50:20 dist-test-slave-1xrh krb5kdc[5717](info): set up 2 sockets
Aug 06 22:50:20 dist-test-slave-1xrh krb5kdc[5717](info): commencing operation
krb5kdc: starting...
W20260806 22:50:21.371208 29547 mini_kdc.cc:121] Time spent starting KDC: real 1.028s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786056586788339-29547-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786056586788339-29547-0/krb5kdc/test-user.keytab.
Aug 06 22:50:21 dist-test-slave-1xrh krb5kdc[5717](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056621, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:50:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:50:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.218.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.218.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.218.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab.
Entry for principal kudu/127.28.218.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab.
I20260806 22:50:21.489279 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786056586788339-29547-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786056586788339-29547-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786056586788339-29547-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.218.254:39633
--webserver_interface=127.28.218.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab
--principal=kudu/127.28.218.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.218.212:44683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.218.254:39633
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:50:21.567838 5730 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:50:21.568033 5730 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:50:21.568054 5730 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:50:21.569118 5730 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:50:21.569149 5730 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:50:21.569164 5730 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:50:21.569176 5730 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:50:21.569188 5730 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:50:21.570531 5730 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:44683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786056586788339-29547-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786056586788339-29547-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.218.254:39633
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.218.254:39633
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.218.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5730
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786056586788339-29547-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:50:21.570708 5730 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:50:21.570892 5730 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:50:21.575652 5736 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:50:21.575714 5730 server_base.cc:1034] running on GCE node
W20260806 22:50:21.575693 5735 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:21.575696 5738 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:50:21.576004 5730 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:50:21.576270 5730 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:50:21.577425 5730 hybrid_clock.cc:648] HybridClock initialized: now 1786056621577411 us; error 28 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-06T22:50:21Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1263 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:50:21 dist-test-slave-1xrh krb5kdc[5743](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:50:21 dist-test-slave-1xrh krb5kdc[5743](info): set up 2 sockets
Aug 06 22:50:21 dist-test-slave-1xrh krb5kdc[5743](info): commencing operation
krb5kdc: starting...
W20260806 22:50:22.646914 29547 mini_kdc.cc:121] Time spent starting KDC: real 1.040s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/krb5kdc/test-user.keytab.
Aug 06 22:50:22 dist-test-slave-1xrh krb5kdc[5743](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056622, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:50:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:50:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.218.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.218.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.218.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab.
Entry for principal kudu/127.28.218.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab.
I20260806 22:50:22.765889 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.218.254:33979
--webserver_interface=127.28.218.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab
--principal=kudu/127.28.218.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.218.212:42965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.218.254:33979
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:50:22.845389 5758 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:50:22.845603 5758 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:50:22.845640 5758 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:50:22.846740 5758 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:50:22.846805 5758 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:50:22.846829 5758 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:50:22.846851 5758 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:50:22.846907 5758 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:50:22.848366 5758 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:42965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.218.254:33979
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab
--principal=kudu/127.28.218.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.218.254:33979
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.218.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5758
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:50:22.848632 5758 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:50:22.848872 5758 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:50:22.853627 5764 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:22.853693 5766 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:22.853621 5763 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:50:22.853885 5758 server_base.cc:1034] running on GCE node
I20260806 22:50:22.854112 5758 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:50:22.854328 5758 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:50:22.855464 5758 hybrid_clock.cc:648] HybridClock initialized: now 1786056622855456 us; error 31 us; skew 500 ppm
Aug 06 22:50:22 dist-test-slave-1xrh krb5kdc[5743](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056622, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:50:22.863035 5758 init.cc:377] Logged in from keytab as kudu/127.28.218.254@KRBTEST.COM (short username kudu)
I20260806 22:50:22.863492 5758 webserver.cc:460] Webserver started at http://127.28.218.254:39045/ using document root <none> and password file <none>
I20260806 22:50:22.863684 5758 fs_manager.cc:359] Metadata directory not provided
I20260806 22:50:22.863724 5758 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:50:22.863821 5758 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:50:22.864771 5758 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/minicluster-data/master-0/data/instance:
uuid: "357341a3088f469c93a97fa39033eb79"
format_stamp: "Formatted at 2026-08-06 22:50:22 on dist-test-slave-1xrh"
server_key: "ac362c03c49446da5b679206c32247fc"
server_key_iv: "ba7f5b78c822342ac49ced0b96c1bacf"
server_key_version: "encryptionkey@0"
I20260806 22:50:22.865065 5758 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/minicluster-data/master-0/wal/instance:
uuid: "357341a3088f469c93a97fa39033eb79"
format_stamp: "Formatted at 2026-08-06 22:50:22 on dist-test-slave-1xrh"
server_key: "ac362c03c49446da5b679206c32247fc"
server_key_iv: "ba7f5b78c822342ac49ced0b96c1bacf"
server_key_version: "encryptionkey@0"
I20260806 22:50:22.866287 5758 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:50:22.866943 5773 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:22.867144 5758 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:50:22.867226 5758 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/minicluster-data/master-0/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/minicluster-data/master-0/wal
uuid: "357341a3088f469c93a97fa39033eb79"
format_stamp: "Formatted at 2026-08-06 22:50:22 on dist-test-slave-1xrh"
server_key: "ac362c03c49446da5b679206c32247fc"
server_key_iv: "ba7f5b78c822342ac49ced0b96c1bacf"
server_key_version: "encryptionkey@0"
I20260806 22:50:22.867278 5758 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:50:22.896338 5758 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:50:22.898511 5758 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:50:22.898630 5758 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:50:22.902362 5758 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.254:33979
I20260806 22:50:22.902454 5827 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.254:33979 every 8 connection(s)
I20260806 22:50:22.902736 5758 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
I20260806 22:50:22.903627 5828 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:50:22.905784 5828 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 357341a3088f469c93a97fa39033eb79: Bootstrap starting.
I20260806 22:50:22.906327 5828 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 357341a3088f469c93a97fa39033eb79: Neither blocks nor log segments found. Creating new log.
I20260806 22:50:22.906608 5828 log.cc:824] T 00000000000000000000000000000000 P 357341a3088f469c93a97fa39033eb79: Log is configured to *not* fsync() on all Append() calls
I20260806 22:50:22.907310 5828 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 357341a3088f469c93a97fa39033eb79: No bootstrap required, opened a new log
I20260806 22:50:22.908833 5828 raft_consensus.cc:348] T 00000000000000000000000000000000 P 357341a3088f469c93a97fa39033eb79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "357341a3088f469c93a97fa39033eb79" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 33979 } }
I20260806 22:50:22.908998 5828 raft_consensus.cc:374] T 00000000000000000000000000000000 P 357341a3088f469c93a97fa39033eb79 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:50:22.909049 5828 raft_consensus.cc:729] T 00000000000000000000000000000000 P 357341a3088f469c93a97fa39033eb79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 357341a3088f469c93a97fa39033eb79, State: Initialized, Role: FOLLOWER
I20260806 22:50:22.909205 5828 consensus_queue.cc:260] T 00000000000000000000000000000000 P 357341a3088f469c93a97fa39033eb79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "357341a3088f469c93a97fa39033eb79" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 33979 } }
I20260806 22:50:22.909299 5828 raft_consensus.cc:388] T 00000000000000000000000000000000 P 357341a3088f469c93a97fa39033eb79 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:50:22.909343 5828 raft_consensus.cc:482] T 00000000000000000000000000000000 P 357341a3088f469c93a97fa39033eb79 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:50:22.909399 5828 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 357341a3088f469c93a97fa39033eb79 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:50:22.909902 5828 raft_consensus.cc:504] T 00000000000000000000000000000000 P 357341a3088f469c93a97fa39033eb79 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "357341a3088f469c93a97fa39033eb79" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 33979 } }
I20260806 22:50:22.910041 5828 leader_election.cc:302] T 00000000000000000000000000000000 P 357341a3088f469c93a97fa39033eb79 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 357341a3088f469c93a97fa39033eb79; no voters:
I20260806 22:50:22.910223 5828 leader_election.cc:288] T 00000000000000000000000000000000 P 357341a3088f469c93a97fa39033eb79 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:50:22.910265 5832 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 357341a3088f469c93a97fa39033eb79 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:50:22.910497 5828 sys_catalog.cc:564] T 00000000000000000000000000000000 P 357341a3088f469c93a97fa39033eb79 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:50:22.910629 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 5758
I20260806 22:50:22.910741 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/minicluster-data/master-0/wal/instance
I20260806 22:50:22.910985 29547 external_mini_cluster.cc:1428] Setting key 861c0629eebe6cf0714db82ce9086dd6
I20260806 22:50:22.911792 5832 raft_consensus.cc:686] T 00000000000000000000000000000000 P 357341a3088f469c93a97fa39033eb79 [term 1 LEADER]: Becoming Leader. State: Replica: 357341a3088f469c93a97fa39033eb79, State: Running, Role: LEADER
I20260806 22:50:22.911896 5832 consensus_queue.cc:237] T 00000000000000000000000000000000 P 357341a3088f469c93a97fa39033eb79 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "357341a3088f469c93a97fa39033eb79" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 33979 } }
I20260806 22:50:22.912369 5833 sys_catalog.cc:455] T 00000000000000000000000000000000 P 357341a3088f469c93a97fa39033eb79 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "357341a3088f469c93a97fa39033eb79" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "357341a3088f469c93a97fa39033eb79" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 33979 } } }
I20260806 22:50:22.912456 5833 sys_catalog.cc:458] T 00000000000000000000000000000000 P 357341a3088f469c93a97fa39033eb79 [sys.catalog]: This master's current role is: LEADER
I20260806 22:50:22.912658 5848 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:50:22.912997 5846 sys_catalog.cc:455] T 00000000000000000000000000000000 P 357341a3088f469c93a97fa39033eb79 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 357341a3088f469c93a97fa39033eb79. Latest consensus state: current_term: 1 leader_uuid: "357341a3088f469c93a97fa39033eb79" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "357341a3088f469c93a97fa39033eb79" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 33979 } } }
I20260806 22:50:22.913085 5846 sys_catalog.cc:458] T 00000000000000000000000000000000 P 357341a3088f469c93a97fa39033eb79 [sys.catalog]: This master's current role is: LEADER
I20260806 22:50:22.914312 5848 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:50:22.915860 5848 catalog_manager.cc:1347] Generated new cluster ID: 4548283c24b6465b8191960f82624925
I20260806 22:50:22.915916 5848 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 22:50:22 dist-test-slave-1xrh krb5kdc[5743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056622, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.218.254@KRBTEST.COM
I20260806 22:50:22.921945 5844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:22.911626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47782 (local address 127.28.218.254:33979)
0806 22:50:22.911789 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:22.911798 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:22.911829 (+ 31us) server_negotiation.cc:407] Connection header received
0806 22:50:22.911937 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:22.911943 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:22.912006 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:22.912148 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:22.912990 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:22.913942 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:22.914935 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:22.915552 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:22.915690 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:22.918469 (+ 2779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:50:22.918472 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:50:22.918473 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:50:22.918483 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:50:22.920348 (+ 1865us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:22.920782 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:22.920785 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:22.920785 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:22.920889 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:22.921101 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:22.921101 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:22.921102 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:22.921366 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:50:22.921502 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:22.921640 (+ 138us) server_negotiation.cc:299] Negotiation successful
0806 22:50:22.921771 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":82,"threads_started":1}
I20260806 22:50:22.924120 5848 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:50:22.924590 5848 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:50:22.933740 5848 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 357341a3088f469c93a97fa39033eb79: Generated new TSK 0
I20260806 22:50:22.933919 5848 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.28.218.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.218.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.218.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.193.keytab.
Entry for principal kudu/127.28.218.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.193.keytab.
I20260806 22:50:22.962458 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.218.193:0
--local_ip_for_outbound_sockets=127.28.218.193
--webserver_interface=127.28.218.193
--webserver_port=0
--tserver_master_addrs=127.28.218.254:33979
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.193.keytab
--principal=kudu/127.28.218.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.218.212:42965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:50:23.066828 5856 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:50:23.067020 5856 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:50:23.067044 5856 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:50:23.068215 5856 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:50:23.068260 5856 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:50:23.068286 5856 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.218.193
I20260806 22:50:23.069746 5856 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:42965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.193.keytab
--principal=kudu/127.28.218.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.218.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.218.193
--webserver_port=0
--tserver_master_addrs=127.28.218.254:33979
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5856
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.218.193
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:50:23.069998 5856 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:50:23.070215 5856 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 22:50:23.075258 5856 server_base.cc:1034] running on GCE node
W20260806 22:50:23.075265 5865 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:23.075222 5862 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:23.075212 5863 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:50:23.075583 5856 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:50:23.075824 5856 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:50:23.076982 5856 hybrid_clock.cc:648] HybridClock initialized: now 1786056623076967 us; error 47 us; skew 500 ppm
Aug 06 22:50:23 dist-test-slave-1xrh krb5kdc[5743](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056623, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:50:23.079381 5856 init.cc:377] Logged in from keytab as kudu/127.28.218.193@KRBTEST.COM (short username kudu)
I20260806 22:50:23.079787 5856 webserver.cc:460] Webserver started at http://127.28.218.193:34579/ using document root <none> and password file <none>
I20260806 22:50:23.080024 5856 fs_manager.cc:359] Metadata directory not provided
I20260806 22:50:23.080094 5856 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:50:23.080248 5856 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:50:23.081260 5856 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/minicluster-data/ts-0/data/instance:
uuid: "7fbcb5743a49408c84d32b3e50aa7212"
format_stamp: "Formatted at 2026-08-06 22:50:23 on dist-test-slave-1xrh"
server_key: "46f746f4ac72ec789d77045322a87004"
server_key_iv: "e7e0ff70b259c6847616295a850d9da2"
server_key_version: "encryptionkey@0"
I20260806 22:50:23.081615 5856 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/minicluster-data/ts-0/wal/instance:
uuid: "7fbcb5743a49408c84d32b3e50aa7212"
format_stamp: "Formatted at 2026-08-06 22:50:23 on dist-test-slave-1xrh"
server_key: "46f746f4ac72ec789d77045322a87004"
server_key_iv: "e7e0ff70b259c6847616295a850d9da2"
server_key_version: "encryptionkey@0"
I20260806 22:50:23.082872 5856 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:50:23.083683 5872 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:23.083873 5856 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:23.083967 5856 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/minicluster-data/ts-0/wal
uuid: "7fbcb5743a49408c84d32b3e50aa7212"
format_stamp: "Formatted at 2026-08-06 22:50:23 on dist-test-slave-1xrh"
server_key: "46f746f4ac72ec789d77045322a87004"
server_key_iv: "e7e0ff70b259c6847616295a850d9da2"
server_key_version: "encryptionkey@0"
I20260806 22:50:23.084059 5856 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:50:23.113412 5856 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:50:23.115198 5856 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:50:23.115347 5856 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:50:23.115559 5856 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:50:23.115847 5856 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:50:23.115899 5856 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:23.115931 5856 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:50:23.115958 5856 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:23.121673 5856 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.193:35341
I20260806 22:50:23.121721 5994 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.193:35341 every 8 connection(s)
I20260806 22:50:23.122037 5856 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/minicluster-data/ts-0/data/info.pb
Aug 06 22:50:23 dist-test-slave-1xrh krb5kdc[5743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056623, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.193@KRBTEST.COM for kudu/127.28.218.254@KRBTEST.COM
I20260806 22:50:23.127874 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 5856
I20260806 22:50:23.127979 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/minicluster-data/ts-0/wal/instance
I20260806 22:50:23.128173 29547 external_mini_cluster.cc:1428] Setting key 6cdd6cde8658c652b75d2e7908825a2e
I20260806 22:50:23.131000 5997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:23.122718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:33979 (local address 127.28.218.193:36115)
0806 22:50:23.122874 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:23.122913 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:23.123422 (+ 509us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:23.123618 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:23.123620 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:23.124155 (+ 535us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:23.124326 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:23.124330 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:23.124961 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:23.124961 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:23.125723 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:23.125725 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:23.126076 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:23.126077 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:23.126172 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:23.126564 (+ 392us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:50:23.126571 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:23.127899 (+ 1328us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:50:23.129587 (+ 1688us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:23.129594 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:23.129596 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:23.129897 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:23.130080 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:23.130081 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:23.130081 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:23.130216 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:23.130475 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:50:23.130481 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:50:23.130653 (+ 172us) client_negotiation.cc:769] Sending connection context
0806 22:50:23.130734 (+ 81us) client_negotiation.cc:240] Negotiation successful
0806 22:50:23.130839 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":91,"threads_started":1}
I20260806 22:50:23.131026 5844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:23.122764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.193:36115 (local address 127.28.218.254:33979)
0806 22:50:23.122814 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:23.122816 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:23.122996 (+ 180us) server_negotiation.cc:407] Connection header received
0806 22:50:23.123504 (+ 508us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:23.123505 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:23.123517 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:23.123570 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:23.124398 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:23.124886 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:23.125792 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:23.126004 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:23.126054 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:23.128018 (+ 1964us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:50:23.128020 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:50:23.128021 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:50:23.128027 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:50:23.129507 (+ 1480us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:23.129967 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:23.129968 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:23.129968 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:23.130019 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:23.130276 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:23.130277 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:23.130277 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:23.130399 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:50:23.130471 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:23.130777 (+ 306us) server_negotiation.cc:299] Negotiation successful
0806 22:50:23.130903 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260806 22:50:23.131747 5995 heartbeater.cc:347] Connected to a master server at 127.28.218.254:33979
I20260806 22:50:23.131839 5995 heartbeater.cc:464] Registering TS with master...
I20260806 22:50:23.132018 5995 heartbeater.cc:511] Master 127.28.218.254:33979 requested a full tablet report, sending...
I20260806 22:50:23.132459 5790 ts_manager.cc:194] Registered new tserver with Master: 7fbcb5743a49408c84d32b3e50aa7212 (127.28.218.193:35341)
I20260806 22:50:23.133080 5790 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.218.193@KRBTEST.COM'} at 127.28.218.193:36115
WARNING: no policy specified for kudu/127.28.218.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.218.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.218.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.194.keytab.
Entry for principal kudu/127.28.218.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.194.keytab.
I20260806 22:50:23.153602 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.218.194:0
--local_ip_for_outbound_sockets=127.28.218.194
--webserver_interface=127.28.218.194
--webserver_port=0
--tserver_master_addrs=127.28.218.254:33979
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.194.keytab
--principal=kudu/127.28.218.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.218.212:42965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:50:23.232045 6000 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:50:23.232262 6000 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:50:23.232302 6000 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:50:23.233428 6000 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:50:23.233494 6000 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:50:23.233546 6000 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.218.194
I20260806 22:50:23.234946 6000 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:42965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.194.keytab
--principal=kudu/127.28.218.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.218.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.218.194
--webserver_port=0
--tserver_master_addrs=127.28.218.254:33979
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6000
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.218.194
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:50:23.235213 6000 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:50:23.235427 6000 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:50:23.240283 6005 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:23.240327 6009 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:23.240283 6006 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:50:23.240507 6000 server_base.cc:1034] running on GCE node
I20260806 22:50:23.240694 6000 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:50:23.240923 6000 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:50:23.242084 6000 hybrid_clock.cc:648] HybridClock initialized: now 1786056623242060 us; error 52 us; skew 500 ppm
Aug 06 22:50:23 dist-test-slave-1xrh krb5kdc[5743](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056623, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:50:23.244494 6000 init.cc:377] Logged in from keytab as kudu/127.28.218.194@KRBTEST.COM (short username kudu)
I20260806 22:50:23.244957 6000 webserver.cc:460] Webserver started at http://127.28.218.194:39159/ using document root <none> and password file <none>
I20260806 22:50:23.245187 6000 fs_manager.cc:359] Metadata directory not provided
I20260806 22:50:23.245252 6000 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:50:23.245381 6000 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:50:23.246343 6000 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/minicluster-data/ts-1/data/instance:
uuid: "6308c4d6e6d6446fb6580e8e72ffc84d"
format_stamp: "Formatted at 2026-08-06 22:50:23 on dist-test-slave-1xrh"
server_key: "05f0375f802c8fce4c9dd92518abf9e7"
server_key_iv: "66350e689b928ef4bb7a3bba5484204b"
server_key_version: "encryptionkey@0"
I20260806 22:50:23.246702 6000 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/minicluster-data/ts-1/wal/instance:
uuid: "6308c4d6e6d6446fb6580e8e72ffc84d"
format_stamp: "Formatted at 2026-08-06 22:50:23 on dist-test-slave-1xrh"
server_key: "05f0375f802c8fce4c9dd92518abf9e7"
server_key_iv: "66350e689b928ef4bb7a3bba5484204b"
server_key_version: "encryptionkey@0"
I20260806 22:50:23.248032 6000 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:50:23.248709 6016 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:23.248871 6000 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:50:23.248967 6000 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/minicluster-data/ts-1/wal
uuid: "6308c4d6e6d6446fb6580e8e72ffc84d"
format_stamp: "Formatted at 2026-08-06 22:50:23 on dist-test-slave-1xrh"
server_key: "05f0375f802c8fce4c9dd92518abf9e7"
server_key_iv: "66350e689b928ef4bb7a3bba5484204b"
server_key_version: "encryptionkey@0"
I20260806 22:50:23.249047 6000 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:50:23.259718 6000 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:50:23.261418 6000 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:50:23.261586 6000 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:50:23.261793 6000 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:50:23.262084 6000 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:50:23.262135 6000 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:23.262176 6000 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:50:23.262202 6000 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:23.267978 6000 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.194:44323
I20260806 22:50:23.268034 6136 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.194:44323 every 8 connection(s)
I20260806 22:50:23.268312 6000 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/minicluster-data/ts-1/data/info.pb
Aug 06 22:50:23 dist-test-slave-1xrh krb5kdc[5743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056623, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.194@KRBTEST.COM for kudu/127.28.218.254@KRBTEST.COM
I20260806 22:50:23.276621 5844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:23.268981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.194:57839 (local address 127.28.218.254:33979)
0806 22:50:23.269032 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:23.269034 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:23.269195 (+ 161us) server_negotiation.cc:407] Connection header received
0806 22:50:23.269624 (+ 429us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:23.269625 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:23.269634 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:23.269683 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:23.270585 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:23.271081 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:23.272064 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:23.272255 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:23.272310 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:23.274235 (+ 1925us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:50:23.274236 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:50:23.274236 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:50:23.274242 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:50:23.275458 (+ 1216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:23.275807 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:23.275807 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:23.275808 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:23.275855 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:23.276087 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:23.276088 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:23.276089 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:23.276204 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:50:23.276270 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:23.276485 (+ 215us) server_negotiation.cc:299] Negotiation successful
0806 22:50:23.276557 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260806 22:50:23.276751 6140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:23.268922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:33979 (local address 127.28.218.194:57839)
0806 22:50:23.269073 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:23.269110 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:23.269537 (+ 427us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:23.269747 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:23.269750 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:23.270309 (+ 559us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:23.270485 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:23.270489 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:23.271208 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:23.271209 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:23.271995 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:23.271997 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:23.272328 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:23.272328 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:23.272417 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:23.272813 (+ 396us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:50:23.272820 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:23.274150 (+ 1330us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:50:23.275534 (+ 1384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:23.275539 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:23.275541 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:23.275745 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:23.275914 (+ 169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:23.275914 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:23.275915 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:23.276005 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:23.276267 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:50:23.276271 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:50:23.276371 (+ 100us) client_negotiation.cc:769] Sending connection context
0806 22:50:23.276455 (+ 84us) client_negotiation.cc:240] Negotiation successful
0806 22:50:23.276564 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":98,"threads_started":1}
I20260806 22:50:23.277253 6138 heartbeater.cc:347] Connected to a master server at 127.28.218.254:33979
I20260806 22:50:23.277372 6138 heartbeater.cc:464] Registering TS with master...
I20260806 22:50:23.277547 6138 heartbeater.cc:511] Master 127.28.218.254:33979 requested a full tablet report, sending...
I20260806 22:50:23.277838 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 6000
I20260806 22:50:23.277961 5790 ts_manager.cc:194] Registered new tserver with Master: 6308c4d6e6d6446fb6580e8e72ffc84d (127.28.218.194:44323)
I20260806 22:50:23.277979 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/minicluster-data/ts-1/wal/instance
I20260806 22:50:23.278154 29547 external_mini_cluster.cc:1428] Setting key 2fda1d75aa06a5e466b7f30f3281d3cd
I20260806 22:50:23.278434 5790 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.218.194@KRBTEST.COM'} at 127.28.218.194:57839
WARNING: no policy specified for kudu/127.28.218.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.218.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.218.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.195.keytab.
Entry for principal kudu/127.28.218.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.195.keytab.
I20260806 22:50:23.303882 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.218.195:0
--local_ip_for_outbound_sockets=127.28.218.195
--webserver_interface=127.28.218.195
--webserver_port=0
--tserver_master_addrs=127.28.218.254:33979
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.195.keytab
--principal=kudu/127.28.218.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.218.212:42965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:50:23.385015 6143 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:50:23.385243 6143 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:50:23.385280 6143 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:50:23.386456 6143 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:50:23.386516 6143 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:50:23.386561 6143 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.218.195
I20260806 22:50:23.388083 6143 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:42965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.195.keytab
--principal=kudu/127.28.218.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.218.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.218.195
--webserver_port=0
--tserver_master_addrs=127.28.218.254:33979
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6143
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.218.195
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:50:23.388324 6143 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:50:23.388562 6143 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:50:23.393368 6148 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:50:23.393484 6143 server_base.cc:1034] running on GCE node
W20260806 22:50:23.393605 6151 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:23.393456 6149 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:50:23.393812 6143 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:50:23.393993 6143 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:50:23.395156 6143 hybrid_clock.cc:648] HybridClock initialized: now 1786056623395144 us; error 26 us; skew 500 ppm
Aug 06 22:50:23 dist-test-slave-1xrh krb5kdc[5743](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056623, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:50:23.397460 6143 init.cc:377] Logged in from keytab as kudu/127.28.218.195@KRBTEST.COM (short username kudu)
I20260806 22:50:23.397886 6143 webserver.cc:460] Webserver started at http://127.28.218.195:34167/ using document root <none> and password file <none>
I20260806 22:50:23.398114 6143 fs_manager.cc:359] Metadata directory not provided
I20260806 22:50:23.398175 6143 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:50:23.398319 6143 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:50:23.399363 6143 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/minicluster-data/ts-2/data/instance:
uuid: "2898ac71031a4782a1f8a370776476af"
format_stamp: "Formatted at 2026-08-06 22:50:23 on dist-test-slave-1xrh"
server_key: "ea987a6f7ce73189e9f2c72a0d1642aa"
server_key_iv: "ba6b295c08abeb4519304fcfd44fa5f3"
server_key_version: "encryptionkey@0"
I20260806 22:50:23.399744 6143 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/minicluster-data/ts-2/wal/instance:
uuid: "2898ac71031a4782a1f8a370776476af"
format_stamp: "Formatted at 2026-08-06 22:50:23 on dist-test-slave-1xrh"
server_key: "ea987a6f7ce73189e9f2c72a0d1642aa"
server_key_iv: "ba6b295c08abeb4519304fcfd44fa5f3"
server_key_version: "encryptionkey@0"
I20260806 22:50:23.401026 6143 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:50:23.401697 6160 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:23.401871 6143 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:50:23.401980 6143 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/minicluster-data/ts-2/wal
uuid: "2898ac71031a4782a1f8a370776476af"
format_stamp: "Formatted at 2026-08-06 22:50:23 on dist-test-slave-1xrh"
server_key: "ea987a6f7ce73189e9f2c72a0d1642aa"
server_key_iv: "ba6b295c08abeb4519304fcfd44fa5f3"
server_key_version: "encryptionkey@0"
I20260806 22:50:23.402064 6143 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:50:23.412088 6143 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:50:23.413843 6143 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:50:23.413990 6143 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:50:23.414179 6143 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:50:23.414453 6143 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:50:23.414502 6143 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:23.414541 6143 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:50:23.414565 6143 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:23.420327 6143 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.195:37255
I20260806 22:50:23.420369 6280 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.195:37255 every 8 connection(s)
I20260806 22:50:23.420650 6143 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/minicluster-data/ts-2/data/info.pb
Aug 06 22:50:23 dist-test-slave-1xrh krb5kdc[5743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056623, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.195@KRBTEST.COM for kudu/127.28.218.254@KRBTEST.COM
I20260806 22:50:23.428296 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 6143
I20260806 22:50:23.428380 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/minicluster-data/ts-2/wal/instance
I20260806 22:50:23.428599 29547 external_mini_cluster.cc:1428] Setting key c0b2504556cd1ba3c3d8ed00273c6880
I20260806 22:50:23.429747 5844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:23.421310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.195:53309 (local address 127.28.218.254:33979)
0806 22:50:23.421363 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:23.421365 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:23.421563 (+ 198us) server_negotiation.cc:407] Connection header received
0806 22:50:23.422048 (+ 485us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:23.422049 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:23.422062 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:23.422112 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:23.422996 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:23.423568 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:23.424566 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:23.424790 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:23.424874 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:23.426840 (+ 1966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:50:23.426841 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:50:23.426842 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:50:23.426850 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:50:23.428129 (+ 1279us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:23.428590 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:23.428592 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:23.428592 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:23.428653 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:23.428989 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:23.428990 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:23.428990 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:23.429164 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:50:23.429275 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:23.429575 (+ 300us) server_negotiation.cc:299] Negotiation successful
0806 22:50:23.429670 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260806 22:50:23.429884 6283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:23.421278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:33979 (local address 127.28.218.195:53309)
0806 22:50:23.421448 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:23.421480 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:23.421936 (+ 456us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:23.422185 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:23.422188 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:23.422726 (+ 538us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:23.422900 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:23.422904 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:23.423668 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:23.423669 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:23.424454 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:23.424456 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:23.424884 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:23.424885 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:23.425000 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:23.425466 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:50:23.425474 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:23.426746 (+ 1272us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:50:23.428219 (+ 1473us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:23.428225 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:23.428227 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:23.428506 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:23.428730 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:23.428731 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:23.428732 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:23.428871 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:23.429243 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:50:23.429249 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:50:23.429413 (+ 164us) client_negotiation.cc:769] Sending connection context
0806 22:50:23.429530 (+ 117us) client_negotiation.cc:240] Negotiation successful
0806 22:50:23.429687 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":83,"threads_started":1}
I20260806 22:50:23.430383 6281 heartbeater.cc:347] Connected to a master server at 127.28.218.254:33979
I20260806 22:50:23.430481 6281 heartbeater.cc:464] Registering TS with master...
I20260806 22:50:23.430619 6281 heartbeater.cc:511] Master 127.28.218.254:33979 requested a full tablet report, sending...
I20260806 22:50:23.430982 5789 ts_manager.cc:194] Registered new tserver with Master: 2898ac71031a4782a1f8a370776476af (127.28.218.195:37255)
I20260806 22:50:23.431396 5789 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.218.195@KRBTEST.COM'} at 127.28.218.195:53309
I20260806 22:50:23.440358 29547 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.28.218.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.28.218.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.28.218.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/krb5kdc/oryx_127.28.218.193.keytab.
Entry for principal oryx/127.28.218.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/krb5kdc/oryx_127.28.218.193.keytab.
WARNING: no policy specified for oryx/127.28.218.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.28.218.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.28.218.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/krb5kdc/oryx_127.28.218.194.keytab.
Entry for principal oryx/127.28.218.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/krb5kdc/oryx_127.28.218.194.keytab.
WARNING: no policy specified for oryx/127.28.218.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.28.218.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.28.218.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/krb5kdc/oryx_127.28.218.195.keytab.
Entry for principal oryx/127.28.218.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/krb5kdc/oryx_127.28.218.195.keytab.
I20260806 22:50:23.506811 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 5856
I20260806 22:50:23.512686 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 6000
I20260806 22:50:23.520164 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 6143
I20260806 22:50:23.525385 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 5758
I20260806 22:50:23.530820 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.218.254:33979
--webserver_interface=127.28.218.254
--webserver_port=39045
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab
--principal=kudu/127.28.218.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.218.212:42965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.218.254:33979
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:50:23.610430 6291 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:50:23.610635 6291 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:50:23.610671 6291 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:50:23.611820 6291 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:50:23.611886 6291 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:50:23.611909 6291 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:50:23.611931 6291 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:50:23.611958 6291 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:50:23.613433 6291 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:42965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.218.254:33979
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab
--principal=kudu/127.28.218.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.218.254:33979
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.218.254
--webserver_port=39045
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6291
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:50:23.613694 6291 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:50:23.613914 6291 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:50:23.618687 6296 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:23.618660 6300 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:23.618705 6297 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:50:23.618899 6291 server_base.cc:1034] running on GCE node
I20260806 22:50:23.619247 6291 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:50:23.619541 6291 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:50:23.620699 6291 hybrid_clock.cc:648] HybridClock initialized: now 1786056623620690 us; error 27 us; skew 500 ppm
Aug 06 22:50:23 dist-test-slave-1xrh krb5kdc[5743](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056623, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:50:23.623174 6291 init.cc:377] Logged in from keytab as kudu/127.28.218.254@KRBTEST.COM (short username kudu)
I20260806 22:50:23.623637 6291 webserver.cc:460] Webserver started at http://127.28.218.254:39045/ using document root <none> and password file <none>
I20260806 22:50:23.623867 6291 fs_manager.cc:359] Metadata directory not provided
I20260806 22:50:23.623955 6291 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:50:23.625277 6291 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:50:23.625941 6307 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:23.626127 6291 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:50:23.626230 6291 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/minicluster-data/master-0/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/minicluster-data/master-0/wal
uuid: "357341a3088f469c93a97fa39033eb79"
format_stamp: "Formatted at 2026-08-06 22:50:22 on dist-test-slave-1xrh"
server_key: "ac362c03c49446da5b679206c32247fc"
server_key_iv: "ba7f5b78c822342ac49ced0b96c1bacf"
server_key_version: "encryptionkey@0"
I20260806 22:50:23.626547 6291 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:50:23.642915 6291 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:50:23.644714 6291 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:50:23.644873 6291 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:50:23.648890 6291 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.254:33979
I20260806 22:50:23.648923 6362 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.254:33979 every 8 connection(s)
I20260806 22:50:23.649279 6291 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
I20260806 22:50:23.650643 6363 sys_catalog.cc:263] Verifying existing consensus state
I20260806 22:50:23.651314 6363 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 357341a3088f469c93a97fa39033eb79: Bootstrap starting.
I20260806 22:50:23.655392 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 6291
I20260806 22:50:23.655681 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.218.193:35341
--local_ip_for_outbound_sockets=127.28.218.193
--tserver_master_addrs=127.28.218.254:33979
--webserver_port=34579
--webserver_interface=127.28.218.193
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.193.keytab
--principal=kudu/127.28.218.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.218.212:42965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.28.218.193
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/krb5kdc/oryx_127.28.218.193.keytab with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:50:23.659359 6363 log.cc:824] T 00000000000000000000000000000000 P 357341a3088f469c93a97fa39033eb79: Log is configured to *not* fsync() on all Append() calls
I20260806 22:50:23.660897 6363 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 357341a3088f469c93a97fa39033eb79: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 22:50:23.661166 6363 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 357341a3088f469c93a97fa39033eb79: Bootstrap complete.
I20260806 22:50:23.663430 6363 raft_consensus.cc:348] T 00000000000000000000000000000000 P 357341a3088f469c93a97fa39033eb79 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "357341a3088f469c93a97fa39033eb79" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 33979 } }
I20260806 22:50:23.663666 6363 raft_consensus.cc:729] T 00000000000000000000000000000000 P 357341a3088f469c93a97fa39033eb79 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 357341a3088f469c93a97fa39033eb79, State: Initialized, Role: FOLLOWER
I20260806 22:50:23.663865 6363 consensus_queue.cc:260] T 00000000000000000000000000000000 P 357341a3088f469c93a97fa39033eb79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "357341a3088f469c93a97fa39033eb79" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 33979 } }
I20260806 22:50:23.663956 6363 raft_consensus.cc:388] T 00000000000000000000000000000000 P 357341a3088f469c93a97fa39033eb79 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:50:23.664009 6363 raft_consensus.cc:482] T 00000000000000000000000000000000 P 357341a3088f469c93a97fa39033eb79 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:50:23.664065 6363 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 357341a3088f469c93a97fa39033eb79 [term 1 FOLLOWER]: Advancing to term 2
I20260806 22:50:23.664973 6363 raft_consensus.cc:504] T 00000000000000000000000000000000 P 357341a3088f469c93a97fa39033eb79 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "357341a3088f469c93a97fa39033eb79" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 33979 } }
I20260806 22:50:23.665093 6363 leader_election.cc:302] T 00000000000000000000000000000000 P 357341a3088f469c93a97fa39033eb79 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 357341a3088f469c93a97fa39033eb79; no voters:
I20260806 22:50:23.665278 6363 leader_election.cc:288] T 00000000000000000000000000000000 P 357341a3088f469c93a97fa39033eb79 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 22:50:23.665311 6367 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 357341a3088f469c93a97fa39033eb79 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 22:50:23.665428 6367 raft_consensus.cc:686] T 00000000000000000000000000000000 P 357341a3088f469c93a97fa39033eb79 [term 2 LEADER]: Becoming Leader. State: Replica: 357341a3088f469c93a97fa39033eb79, State: Running, Role: LEADER
I20260806 22:50:23.665535 6363 sys_catalog.cc:564] T 00000000000000000000000000000000 P 357341a3088f469c93a97fa39033eb79 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:50:23.665525 6367 consensus_queue.cc:237] T 00000000000000000000000000000000 P 357341a3088f469c93a97fa39033eb79 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "357341a3088f469c93a97fa39033eb79" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 33979 } }
I20260806 22:50:23.665910 6370 sys_catalog.cc:455] T 00000000000000000000000000000000 P 357341a3088f469c93a97fa39033eb79 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 357341a3088f469c93a97fa39033eb79. Latest consensus state: current_term: 2 leader_uuid: "357341a3088f469c93a97fa39033eb79" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "357341a3088f469c93a97fa39033eb79" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 33979 } } }
I20260806 22:50:23.666162 6370 sys_catalog.cc:458] T 00000000000000000000000000000000 P 357341a3088f469c93a97fa39033eb79 [sys.catalog]: This master's current role is: LEADER
I20260806 22:50:23.665916 6368 sys_catalog.cc:455] T 00000000000000000000000000000000 P 357341a3088f469c93a97fa39033eb79 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "357341a3088f469c93a97fa39033eb79" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "357341a3088f469c93a97fa39033eb79" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 33979 } } }
I20260806 22:50:23.666262 6368 sys_catalog.cc:458] T 00000000000000000000000000000000 P 357341a3088f469c93a97fa39033eb79 [sys.catalog]: This master's current role is: LEADER
I20260806 22:50:23.666373 6377 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:50:23.666878 6377 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:50:23.667024 6377 catalog_manager.cc:1259] Loaded cluster ID: 4548283c24b6465b8191960f82624925
I20260806 22:50:23.667063 6377 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:50:23.668208 6377 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:50:23.668385 6377 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 357341a3088f469c93a97fa39033eb79: Loaded TSK: 0
I20260806 22:50:23.668618 6377 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 22:50:23.737216 6365 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:50:23.737442 6365 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:50:23.737480 6365 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:50:23.738590 6365 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:50:23.738651 6365 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:50:23.738692 6365 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.218.193
I20260806 22:50:23.740048 6365 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:42965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/krb5kdc/oryx_127.28.218.193.keytab
--principal=oryx/127.28.218.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.218.193:35341
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.218.193
--webserver_port=34579
--tserver_master_addrs=127.28.218.254:33979
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6365
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.218.193
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:50:23.740286 6365 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:50:23.740496 6365 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:50:23.746508 6393 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:23.746557 6390 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:23.746667 6391 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:50:23.746898 6365 server_base.cc:1034] running on GCE node
I20260806 22:50:23.747108 6365 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:50:23.747412 6365 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:50:23.748574 6365 hybrid_clock.cc:648] HybridClock initialized: now 1786056623748563 us; error 27 us; skew 500 ppm
Aug 06 22:50:23 dist-test-slave-1xrh krb5kdc[5743](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056623, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.218.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:50:23.751021 6365 init.cc:377] Logged in from keytab as oryx/127.28.218.193@KRBTEST.COM (short username oryx)
I20260806 22:50:23.751562 6365 webserver.cc:460] Webserver started at http://127.28.218.193:34579/ using document root <none> and password file <none>
I20260806 22:50:23.751791 6365 fs_manager.cc:359] Metadata directory not provided
I20260806 22:50:23.751849 6365 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:50:23.753243 6365 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:50:23.753883 6401 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:23.754029 6365 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:50:23.754108 6365 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/minicluster-data/ts-0/wal
uuid: "7fbcb5743a49408c84d32b3e50aa7212"
format_stamp: "Formatted at 2026-08-06 22:50:23 on dist-test-slave-1xrh"
server_key: "46f746f4ac72ec789d77045322a87004"
server_key_iv: "e7e0ff70b259c6847616295a850d9da2"
server_key_version: "encryptionkey@0"
I20260806 22:50:23.754456 6365 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:50:23.785635 6365 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:50:23.788057 6365 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:50:23.788229 6365 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:50:23.788486 6365 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:50:23.788810 6365 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:50:23.788843 6365 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:23.788866 6365 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:50:23.788910 6365 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:23.794888 6365 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.193:35341
I20260806 22:50:23.794945 6525 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.193:35341 every 8 connection(s)
I20260806 22:50:23.795291 6365 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/minicluster-data/ts-0/data/info.pb
I20260806 22:50:23.801151 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 6365
I20260806 22:50:23.801470 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.218.194:44323
--local_ip_for_outbound_sockets=127.28.218.194
--tserver_master_addrs=127.28.218.254:33979
--webserver_port=39159
--webserver_interface=127.28.218.194
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.194.keytab
--principal=kudu/127.28.218.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.218.212:42965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.28.218.194
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/krb5kdc/oryx_127.28.218.194.keytab with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 22:50:23 dist-test-slave-1xrh krb5kdc[5743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.218.193@KRBTEST.COM for oryx/127.28.218.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:50:23 dist-test-slave-1xrh krb5kdc[5743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.218.193@KRBTEST.COM for oryx/127.28.218.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:50:23.803256 6528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:23.795982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:33979 (local address 127.28.218.193:47199)
0806 22:50:23.796141 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:23.796184 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:23.796643 (+ 459us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:23.797020 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:23.797023 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:23.797564 (+ 541us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:23.797749 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:23.797754 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:23.798775 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:23.798776 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:23.799614 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:23.799617 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:23.800153 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:23.800154 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:23.800245 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:23.800661 (+ 416us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:50:23.800669 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:23.802991 (+ 2322us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.218.254:33979: Server oryx/127.28.218.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":87,"threads_started":1}
W20260806 22:50:23.803386 6529 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:50:23.796067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.193:47199 (local address 127.28.218.254:33979)
0806 22:50:23.796251 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:23.796259 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:23.796308 (+ 49us) server_negotiation.cc:407] Connection header received
0806 22:50:23.796804 (+ 496us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:23.796809 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:23.796849 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:23.796964 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:23.797850 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:23.798691 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:23.799691 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:23.800072 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:23.800155 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:23.803224 (+ 3069us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.218.193:47199: BlockingRecv error: recv got EOF from 127.28.218.193:47199 (error 108)
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":89,"threads_started":1}
W20260806 22:50:23.803452 6526 heartbeater.cc:644] Failed to heartbeat to 127.28.218.254:33979 (0 consecutive failures): Not authorized: Failed to ping master at 127.28.218.254:33979: Client connection negotiation failed: client connection to 127.28.218.254:33979: Server oryx/127.28.218.254@KRBTEST.COM not found in Kerberos database
Aug 06 22:50:23 dist-test-slave-1xrh krb5kdc[5743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.218.193@KRBTEST.COM for oryx/127.28.218.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:50:23 dist-test-slave-1xrh krb5kdc[5743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.218.193@KRBTEST.COM for oryx/127.28.218.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:50:23.808027 6528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:23.803671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:33979 (local address 127.28.218.193:53727)
0806 22:50:23.803777 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:23.803787 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:23.803871 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:23.804006 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:23.804007 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:23.804141 (+ 134us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:23.804183 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:23.804184 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:23.804788 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:23.804789 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:23.805327 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:23.805328 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:23.805722 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:23.805723 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:23.805790 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:23.806196 (+ 406us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:50:23.806201 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:23.807937 (+ 1736us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.218.254:33979: Server oryx/127.28.218.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":87}
W20260806 22:50:23.808054 6529 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:50:23.803732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.193:53727 (local address 127.28.218.254:33979)
0806 22:50:23.803765 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:23.803767 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:23.803864 (+ 97us) server_negotiation.cc:407] Connection header received
0806 22:50:23.803936 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:23.803936 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:23.803947 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:23.804003 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:23.804265 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:23.804711 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:23.805407 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:23.805651 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:23.805724 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:23.808005 (+ 2281us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.218.193:53727: BlockingRecv error: recv got EOF from 127.28.218.193:53727 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
Aug 06 22:50:23 dist-test-slave-1xrh krb5kdc[5743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.218.193@KRBTEST.COM for oryx/127.28.218.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:50:23 dist-test-slave-1xrh krb5kdc[5743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.218.193@KRBTEST.COM for oryx/127.28.218.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:50:23.812381 6528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:23.808272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:33979 (local address 127.28.218.193:32817)
0806 22:50:23.808389 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:23.808401 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:23.808451 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:23.808598 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:23.808599 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:23.808737 (+ 138us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:23.808759 (+ 22us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:23.808760 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:23.809341 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:23.809342 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:23.809822 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:23.809824 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:23.810165 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:23.810166 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:23.810217 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:23.810636 (+ 419us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:50:23.810640 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:23.812308 (+ 1668us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.218.254:33979: Server oryx/127.28.218.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":75,"threads_started":1}
W20260806 22:50:23.812431 6529 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:50:23.808327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.193:32817 (local address 127.28.218.254:33979)
0806 22:50:23.808360 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:23.808362 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:23.808453 (+ 91us) server_negotiation.cc:407] Connection header received
0806 22:50:23.808510 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:23.808511 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:23.808525 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:23.808592 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:23.808830 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:23.809248 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:23.809903 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:23.810096 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:23.810166 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:23.812374 (+ 2208us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.218.193:32817: BlockingRecv error: recv got EOF from 127.28.218.193:32817 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260806 22:50:23.812518 6526 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 22:50:23.884428 6530 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:50:23.884634 6530 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:50:23.884671 6530 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:50:23.885784 6530 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:50:23.885847 6530 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:50:23.885893 6530 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.218.194
I20260806 22:50:23.887372 6530 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:42965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/krb5kdc/oryx_127.28.218.194.keytab
--principal=oryx/127.28.218.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.218.194:44323
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.218.194
--webserver_port=39159
--tserver_master_addrs=127.28.218.254:33979
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6530
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.218.194
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:50:23.887622 6530 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:50:23.887838 6530 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:50:23.892606 6540 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:23.892627 6538 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:23.892601 6536 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:50:23.892642 6530 server_base.cc:1034] running on GCE node
I20260806 22:50:23.892972 6530 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:50:23.893217 6530 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:50:23.894372 6530 hybrid_clock.cc:648] HybridClock initialized: now 1786056623894355 us; error 46 us; skew 500 ppm
Aug 06 22:50:23 dist-test-slave-1xrh krb5kdc[5743](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056623, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.218.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:50:23.896943 6530 init.cc:377] Logged in from keytab as oryx/127.28.218.194@KRBTEST.COM (short username oryx)
I20260806 22:50:23.897338 6530 webserver.cc:460] Webserver started at http://127.28.218.194:39159/ using document root <none> and password file <none>
I20260806 22:50:23.897575 6530 fs_manager.cc:359] Metadata directory not provided
I20260806 22:50:23.897680 6530 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:50:23.898939 6530 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:50:23.899555 6547 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:23.899772 6530 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:23.899864 6530 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/minicluster-data/ts-1/wal
uuid: "6308c4d6e6d6446fb6580e8e72ffc84d"
format_stamp: "Formatted at 2026-08-06 22:50:23 on dist-test-slave-1xrh"
server_key: "05f0375f802c8fce4c9dd92518abf9e7"
server_key_iv: "66350e689b928ef4bb7a3bba5484204b"
server_key_version: "encryptionkey@0"
I20260806 22:50:23.900166 6530 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:50:23.908557 6530 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:50:23.910190 6530 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:50:23.910341 6530 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:50:23.910557 6530 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:50:23.910832 6530 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:50:23.910884 6530 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:23.910915 6530 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:50:23.910941 6530 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:23.916595 6530 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.194:44323
I20260806 22:50:23.916630 6667 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.194:44323 every 8 connection(s)
I20260806 22:50:23.916947 6530 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/minicluster-data/ts-1/data/info.pb
Aug 06 22:50:23 dist-test-slave-1xrh krb5kdc[5743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.218.194@KRBTEST.COM for oryx/127.28.218.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:50:23 dist-test-slave-1xrh krb5kdc[5743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.218.194@KRBTEST.COM for oryx/127.28.218.254@KRBTEST.COM, Server not found in Kerberos database
W20260806 22:50:23.924206 6529 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:50:23.917621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.194:38023 (local address 127.28.218.254:33979)
0806 22:50:23.917691 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:23.917693 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:23.917905 (+ 212us) server_negotiation.cc:407] Connection header received
0806 22:50:23.918384 (+ 479us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:23.918385 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:23.918402 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:23.918454 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:23.919389 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:23.919881 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:23.920847 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:23.921048 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:23.921124 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:23.924120 (+ 2996us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.218.194:38023: BlockingRecv error: recv got EOF from 127.28.218.194:38023 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 22:50:23.924228 6670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:23.917624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:33979 (local address 127.28.218.194:38023)
0806 22:50:23.917795 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:23.917826 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:23.918274 (+ 448us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:23.918520 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:23.918523 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:23.919045 (+ 522us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:23.919292 (+ 247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:23.919297 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:23.919986 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:23.919987 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:23.920751 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:23.920753 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:23.921128 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:23.921129 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:23.921249 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:23.921722 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:50:23.921730 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:23.924033 (+ 2303us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.218.254:33979: Server oryx/127.28.218.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":88,"threads_started":1}
W20260806 22:50:23.924485 6668 heartbeater.cc:644] Failed to heartbeat to 127.28.218.254:33979 (0 consecutive failures): Not authorized: Failed to ping master at 127.28.218.254:33979: Client connection negotiation failed: client connection to 127.28.218.254:33979: Server oryx/127.28.218.254@KRBTEST.COM not found in Kerberos database
I20260806 22:50:23.926965 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 6530
I20260806 22:50:23.927301 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.218.195:37255
--local_ip_for_outbound_sockets=127.28.218.195
--tserver_master_addrs=127.28.218.254:33979
--webserver_port=34167
--webserver_interface=127.28.218.195
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.195.keytab
--principal=kudu/127.28.218.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.218.212:42965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.28.218.195
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/krb5kdc/oryx_127.28.218.195.keytab with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 22:50:23 dist-test-slave-1xrh krb5kdc[5743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.218.194@KRBTEST.COM for oryx/127.28.218.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:50:23 dist-test-slave-1xrh krb5kdc[5743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.218.194@KRBTEST.COM for oryx/127.28.218.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:50:23.928515 6670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:23.924697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:33979 (local address 127.28.218.194:47401)
0806 22:50:23.924747 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:23.924758 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:23.924793 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:23.924899 (+ 106us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:23.924900 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:23.925001 (+ 101us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:23.925043 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:23.925044 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:23.925575 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:23.925576 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:23.926045 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:23.926046 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:23.926335 (+ 289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:23.926336 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:23.926382 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:23.926687 (+ 305us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:50:23.926691 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:23.928441 (+ 1750us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.218.254:33979: Server oryx/127.28.218.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":34}
W20260806 22:50:23.928584 6529 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:50:23.924762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.194:47401 (local address 127.28.218.254:33979)
0806 22:50:23.924795 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:23.924797 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:23.924805 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:50:23.924851 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:23.924851 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:23.924858 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:23.924895 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:23.925091 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:23.925517 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:23.926111 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:23.926276 (+ 165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:23.926323 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:23.928507 (+ 2184us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.218.194:47401: BlockingRecv error: recv got EOF from 127.28.218.194:47401 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
Aug 06 22:50:23 dist-test-slave-1xrh krb5kdc[5743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.218.194@KRBTEST.COM for oryx/127.28.218.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:50:23 dist-test-slave-1xrh krb5kdc[5743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.218.194@KRBTEST.COM for oryx/127.28.218.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:50:23.932974 6670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:23.928832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:33979 (local address 127.28.218.194:58611)
0806 22:50:23.928904 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:23.928917 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:23.928966 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:23.929116 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:23.929117 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:23.929234 (+ 117us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:23.929258 (+ 24us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:23.929259 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:23.929866 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:23.929867 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:23.930403 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:23.930404 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:23.930764 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:23.930765 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:23.930816 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:23.931209 (+ 393us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:50:23.931214 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:23.932902 (+ 1688us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.218.254:33979: Server oryx/127.28.218.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":51}
W20260806 22:50:23.933028 6529 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:50:23.928893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.194:58611 (local address 127.28.218.254:33979)
0806 22:50:23.928944 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:23.928946 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:23.928954 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:50:23.929029 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:23.929030 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:23.929037 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:23.929098 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:23.929329 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:23.929789 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:23.930484 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:23.930680 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:23.930763 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:23.932968 (+ 2205us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.218.194:58611: BlockingRecv error: recv got EOF from 127.28.218.194:58611 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
W20260806 22:50:23.933108 6668 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 22:50:24.010332 6671 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:50:24.010550 6671 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:50:24.010584 6671 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:50:24.011718 6671 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:50:24.011780 6671 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:50:24.011822 6671 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.218.195
I20260806 22:50:24.013229 6671 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:42965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/krb5kdc/oryx_127.28.218.195.keytab
--principal=oryx/127.28.218.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.218.195:37255
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.218.195
--webserver_port=34167
--tserver_master_addrs=127.28.218.254:33979
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6671
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.218.195
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:50:24.013481 6671 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:50:24.013697 6671 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:50:24.018512 6677 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:24.018522 6680 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:24.018635 6676 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:50:24.019061 6671 server_base.cc:1034] running on GCE node
I20260806 22:50:24.019277 6671 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:50:24.019519 6671 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:50:24.020673 6671 hybrid_clock.cc:648] HybridClock initialized: now 1786056624020669 us; error 35 us; skew 500 ppm
Aug 06 22:50:24 dist-test-slave-1xrh krb5kdc[5743](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056624, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.218.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:50:24.023109 6671 init.cc:377] Logged in from keytab as oryx/127.28.218.195@KRBTEST.COM (short username oryx)
I20260806 22:50:24.023571 6671 webserver.cc:460] Webserver started at http://127.28.218.195:34167/ using document root <none> and password file <none>
I20260806 22:50:24.023819 6671 fs_manager.cc:359] Metadata directory not provided
I20260806 22:50:24.023903 6671 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:50:24.025161 6671 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:50:24.025735 6687 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:24.025899 6671 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:50:24.025990 6671 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/minicluster-data/ts-2/wal
uuid: "2898ac71031a4782a1f8a370776476af"
format_stamp: "Formatted at 2026-08-06 22:50:23 on dist-test-slave-1xrh"
server_key: "ea987a6f7ce73189e9f2c72a0d1642aa"
server_key_iv: "ba6b295c08abeb4519304fcfd44fa5f3"
server_key_version: "encryptionkey@0"
I20260806 22:50:24.026307 6671 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:50:24.042927 6671 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:50:24.044802 6671 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:50:24.044965 6671 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:50:24.045202 6671 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:50:24.045493 6671 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:50:24.045544 6671 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:24.045581 6671 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:50:24.045608 6671 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:24.051450 6671 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.195:37255
I20260806 22:50:24.051496 6808 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.195:37255 every 8 connection(s)
I20260806 22:50:24.051863 6671 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056586788339-29547-0/minicluster-data/ts-2/data/info.pb
I20260806 22:50:24.053028 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 6671
Aug 06 22:50:24 dist-test-slave-1xrh krb5kdc[5743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.218.195@KRBTEST.COM for oryx/127.28.218.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:50:24 dist-test-slave-1xrh krb5kdc[5743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.218.195@KRBTEST.COM for oryx/127.28.218.254@KRBTEST.COM, Server not found in Kerberos database
W20260806 22:50:24.059538 6529 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:50:24.052623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.195:42079 (local address 127.28.218.254:33979)
0806 22:50:24.052689 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:24.052691 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:24.052884 (+ 193us) server_negotiation.cc:407] Connection header received
0806 22:50:24.053468 (+ 584us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:24.053469 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:24.053482 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:24.053536 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:24.054455 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:24.054967 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:24.056010 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:24.056193 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:24.056274 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:24.059459 (+ 3185us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.218.195:42079: BlockingRecv error: recv got EOF from 127.28.218.195:42079 (error 108)
Metrics: {"mutex_wait_us":28,"server-negotiator.queue_time_us":37}
I20260806 22:50:24.059608 6811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:24.052593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:33979 (local address 127.28.218.195:42079)
0806 22:50:24.052770 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:24.052805 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:24.053351 (+ 546us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:24.053601 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:24.053605 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:24.054169 (+ 564us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:24.054345 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:24.054349 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:24.055064 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:24.055065 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:24.055899 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:24.055901 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:24.056287 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:24.056288 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:24.056417 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:24.056797 (+ 380us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:50:24.056804 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:24.059385 (+ 2581us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.218.254:33979: Server oryx/127.28.218.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":90,"threads_started":1}
W20260806 22:50:24.059808 6809 heartbeater.cc:644] Failed to heartbeat to 127.28.218.254:33979 (0 consecutive failures): Not authorized: Failed to ping master at 127.28.218.254:33979: Client connection negotiation failed: client connection to 127.28.218.254:33979: Server oryx/127.28.218.254@KRBTEST.COM not found in Kerberos database
I20260806 22:50:24.060937 6813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:24.053538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47792 (local address 127.28.218.254:33979)
0806 22:50:24.053656 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:24.053658 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:24.053738 (+ 80us) server_negotiation.cc:407] Connection header received
0806 22:50:24.053814 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:24.053815 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:24.053827 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:24.053884 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:24.054470 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:24.054969 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:24.055792 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:24.056010 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:24.056090 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:24.057434 (+ 1344us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:50:24.057438 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:50:24.057439 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:50:24.057450 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:50:24.059448 (+ 1998us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:24.059850 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:24.059853 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:24.059854 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:24.060004 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:24.060222 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:24.060223 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:24.060223 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:24.060504 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:50:24.060654 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:24.060741 (+ 87us) server_negotiation.cc:299] Negotiation successful
0806 22:50:24.060844 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":97,"thread_start_us":65,"threads_started":1}
Aug 06 22:50:24 dist-test-slave-1xrh krb5kdc[5743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.218.195@KRBTEST.COM for oryx/127.28.218.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:50:24 dist-test-slave-1xrh krb5kdc[5743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.218.195@KRBTEST.COM for oryx/127.28.218.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:50:24.064214 6811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:24.060017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:33979 (local address 127.28.218.195:59449)
0806 22:50:24.060056 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:24.060065 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:24.060141 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:24.060278 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:24.060278 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:24.060378 (+ 100us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:24.060424 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:24.060425 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:24.061235 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:24.061236 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:24.061946 (+ 710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:24.061948 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:24.062227 (+ 279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:24.062227 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:24.062273 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:24.062644 (+ 371us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:50:24.062649 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:24.064175 (+ 1526us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.218.254:33979: Server oryx/127.28.218.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":18}
W20260806 22:50:24.064323 6529 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:50:24.060146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.195:59449 (local address 127.28.218.254:33979)
0806 22:50:24.060184 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:24.060186 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:24.060193 (+ 7us) server_negotiation.cc:407] Connection header received
0806 22:50:24.060211 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:24.060212 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:24.060220 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:24.060274 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:24.060490 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:24.061140 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:24.062003 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:24.062168 (+ 165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:24.062218 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:24.064252 (+ 2034us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.218.195:59449: BlockingRecv error: recv got EOF from 127.28.218.195:59449 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
Aug 06 22:50:24 dist-test-slave-1xrh krb5kdc[5743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.218.195@KRBTEST.COM for oryx/127.28.218.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:50:24 dist-test-slave-1xrh krb5kdc[5743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.218.195@KRBTEST.COM for oryx/127.28.218.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:50:24.068022 6811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:24.064476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:33979 (local address 127.28.218.195:51893)
0806 22:50:24.064546 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:24.064557 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:24.064595 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:24.064690 (+ 95us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:24.064691 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:24.064788 (+ 97us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:24.064812 (+ 24us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:24.064812 (+ 0us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:24.065329 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:24.065329 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:24.065870 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:24.065872 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:24.066146 (+ 274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:24.066147 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:24.066178 (+ 31us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:24.066467 (+ 289us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:50:24.066470 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:24.067983 (+ 1513us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.218.254:33979: Server oryx/127.28.218.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":54}
W20260806 22:50:24.068150 6809 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 22:50:24.068157 6529 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:50:24.064533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.195:51893 (local address 127.28.218.254:33979)
0806 22:50:24.064570 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:24.064572 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:24.064591 (+ 19us) server_negotiation.cc:407] Connection header received
0806 22:50:24.064644 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:24.064645 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:24.064651 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:24.064687 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:24.064864 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:24.065270 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:24.065931 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:24.066089 (+ 158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:24.066136 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:24.068073 (+ 1937us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.218.195:51893: BlockingRecv error: recv got EOF from 127.28.218.195:51893 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
Aug 06 22:50:24 dist-test-slave-1xrh krb5kdc[5743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.218.193@KRBTEST.COM for oryx/127.28.218.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:50:24 dist-test-slave-1xrh krb5kdc[5743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.218.193@KRBTEST.COM for oryx/127.28.218.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:50:24.817288 6814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:24.812848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:33979 (local address 127.28.218.193:59121)
0806 22:50:24.812993 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:24.813005 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:24.813064 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:24.813199 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:24.813199 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:24.813362 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:24.813411 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:24.813412 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:24.814050 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:24.814051 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:24.814582 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:24.814584 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:24.814899 (+ 315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:24.814900 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:24.814937 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:24.815381 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:50:24.815386 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:24.817219 (+ 1833us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.218.254:33979: Server oryx/127.28.218.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":92,"threads_started":1}
W20260806 22:50:24.817395 6815 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:50:24.812940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.193:59121 (local address 127.28.218.254:33979)
0806 22:50:24.813059 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:24.813061 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:24.813070 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:50:24.813119 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:24.813120 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:24.813129 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:24.813185 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:24.813475 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:24.813984 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:24.814652 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:24.814838 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:24.814886 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:24.817309 (+ 2423us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.218.193:59121: BlockingRecv error: recv got EOF from 127.28.218.193:59121 (error 108)
Metrics: {"server-negotiator.queue_time_us":90,"thread_start_us":61,"threads_started":1}
Aug 06 22:50:24 dist-test-slave-1xrh krb5kdc[5743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.218.194@KRBTEST.COM for oryx/127.28.218.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:50:24 dist-test-slave-1xrh krb5kdc[5743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.218.194@KRBTEST.COM for oryx/127.28.218.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:50:24.937830 6816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:24.933488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:33979 (local address 127.28.218.194:55773)
0806 22:50:24.933652 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:24.933662 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:24.933721 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:24.933867 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:24.933867 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:24.934033 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:24.934082 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:24.934084 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:24.934685 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:24.934686 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:24.935237 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:24.935238 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:24.935549 (+ 311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:24.935550 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:24.935589 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:24.936013 (+ 424us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:50:24.936018 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:24.937765 (+ 1747us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.218.254:33979: Server oryx/127.28.218.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":103,"threads_started":1}
W20260806 22:50:24.937923 6815 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:50:24.933540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.194:55773 (local address 127.28.218.254:33979)
0806 22:50:24.933612 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:24.933614 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:24.933723 (+ 109us) server_negotiation.cc:407] Connection header received
0806 22:50:24.933789 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:24.933789 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:24.933800 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:24.933848 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:24.934146 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:24.934624 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:24.935302 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:24.935480 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:24.935530 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:24.937856 (+ 2326us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.218.194:55773: BlockingRecv error: recv got EOF from 127.28.218.194:55773 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 06 22:50:25 dist-test-slave-1xrh krb5kdc[5743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.218.195@KRBTEST.COM for oryx/127.28.218.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:50:25 dist-test-slave-1xrh krb5kdc[5743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.218.195@KRBTEST.COM for oryx/127.28.218.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:50:25.073493 6817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:25.068478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:33979 (local address 127.28.218.195:34919)
0806 22:50:25.068610 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:25.068620 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:25.068671 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:25.068836 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:25.068838 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:25.069026 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:25.069080 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:25.069081 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:25.070043 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:25.070044 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:25.070572 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:25.070573 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:25.070891 (+ 318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:25.070892 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:25.070930 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:25.071459 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:50:25.071464 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:25.073378 (+ 1914us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.218.254:33979: Server oryx/127.28.218.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":81,"threads_started":1}
W20260806 22:50:25.073527 6815 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:50:25.068528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.195:34919 (local address 127.28.218.254:33979)
0806 22:50:25.068595 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:25.068597 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:25.068672 (+ 75us) server_negotiation.cc:407] Connection header received
0806 22:50:25.068713 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:25.068714 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:25.068726 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:25.068776 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:25.069211 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:25.069975 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:25.070638 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:25.070815 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:25.070889 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:25.073461 (+ 2572us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.218.195:34919: BlockingRecv error: recv got EOF from 127.28.218.195:34919 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Aug 06 22:50:25 dist-test-slave-1xrh krb5kdc[5743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.218.193@KRBTEST.COM for oryx/127.28.218.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:50:25 dist-test-slave-1xrh krb5kdc[5743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.218.193@KRBTEST.COM for oryx/127.28.218.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:50:25.822494 6818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:25.817801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:33979 (local address 127.28.218.193:33445)
0806 22:50:25.817965 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:25.817976 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:25.818030 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:25.818217 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:25.818218 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:25.818384 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:25.818435 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:25.818436 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:25.819068 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:25.819068 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:25.819650 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:25.819652 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:25.820015 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:25.820016 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:25.820084 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:25.820618 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:50:25.820623 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:25.822432 (+ 1809us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.218.254:33979: Server oryx/127.28.218.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":107,"threads_started":1}
W20260806 22:50:25.822624 6820 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:50:25.817850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.193:33445 (local address 127.28.218.254:33979)
0806 22:50:25.817987 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:25.817989 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:25.818017 (+ 28us) server_negotiation.cc:407] Connection header received
0806 22:50:25.818085 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:25.818086 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:25.818098 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:25.818162 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:25.818507 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:25.819002 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:25.819735 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:25.819932 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:25.819983 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:25.822520 (+ 2537us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.218.193:33445: BlockingRecv error: recv got EOF from 127.28.218.193:33445 (error 108)
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":80,"threads_started":1}
Aug 06 22:50:25 dist-test-slave-1xrh krb5kdc[5743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.218.194@KRBTEST.COM for oryx/127.28.218.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:50:25 dist-test-slave-1xrh krb5kdc[5743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.218.194@KRBTEST.COM for oryx/127.28.218.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:50:25.943198 6821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:25.938350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:33979 (local address 127.28.218.194:45531)
0806 22:50:25.938503 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:25.938513 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:25.938569 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:25.938701 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:25.938702 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:25.938880 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:25.938929 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:25.938931 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:25.939665 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:25.939666 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:25.940230 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:25.940231 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:25.940558 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:25.940559 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:25.940603 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:25.941032 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:50:25.941036 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:25.943075 (+ 2039us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.218.254:33979: Server oryx/127.28.218.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":87,"threads_started":1}
W20260806 22:50:25.943250 6820 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:50:25.938444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.194:45531 (local address 127.28.218.254:33979)
0806 22:50:25.938508 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:25.938510 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:25.938569 (+ 59us) server_negotiation.cc:407] Connection header received
0806 22:50:25.938630 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:25.938630 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:25.938640 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:25.938683 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:25.938994 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:25.939543 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:25.940303 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:25.940487 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:25.940536 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:25.943184 (+ 2648us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.218.194:45531: BlockingRecv error: recv got EOF from 127.28.218.194:45531 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Aug 06 22:50:26 dist-test-slave-1xrh krb5kdc[5743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.218.195@KRBTEST.COM for oryx/127.28.218.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:50:26 dist-test-slave-1xrh krb5kdc[5743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.218.195@KRBTEST.COM for oryx/127.28.218.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:50:26.078680 6822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:26.073961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:33979 (local address 127.28.218.195:40095)
0806 22:50:26.074145 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:26.074159 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:26.074225 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:26.074364 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:26.074365 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:26.074558 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:26.074607 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:26.074608 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:26.075331 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:26.075332 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:26.075853 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:26.075855 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:26.076218 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:26.076219 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:26.076277 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:26.076778 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:50:26.076783 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:26.078624 (+ 1841us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.218.254:33979: Server oryx/127.28.218.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":77,"threads_started":1}
W20260806 22:50:26.078785 6820 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:50:26.074074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.195:40095 (local address 127.28.218.254:33979)
0806 22:50:26.074156 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:26.074159 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:26.074206 (+ 47us) server_negotiation.cc:407] Connection header received
0806 22:50:26.074292 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:26.074293 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:26.074307 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:26.074359 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:26.074687 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:26.075252 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:26.075938 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:26.076145 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:26.076220 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:26.078714 (+ 2494us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.218.195:40095: BlockingRecv error: recv got EOF from 127.28.218.195:40095 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 06 22:50:26 dist-test-slave-1xrh krb5kdc[5743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.218.193@KRBTEST.COM for oryx/127.28.218.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:50:26 dist-test-slave-1xrh krb5kdc[5743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.218.193@KRBTEST.COM for oryx/127.28.218.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:50:26.828019 6823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:26.823012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:33979 (local address 127.28.218.193:40451)
0806 22:50:26.823191 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:26.823210 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:26.823263 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:26.823419 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:26.823420 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:26.823590 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:26.823674 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:26.823677 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:26.824585 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:26.824586 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:26.825131 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:26.825133 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:26.825508 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:26.825510 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:26.825551 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:26.825993 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:50:26.825998 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:26.827950 (+ 1952us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.218.254:33979: Server oryx/127.28.218.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":125,"threads_started":1}
W20260806 22:50:26.828156 6824 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:50:26.823075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.193:40451 (local address 127.28.218.254:33979)
0806 22:50:26.823264 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:26.823266 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:26.823299 (+ 33us) server_negotiation.cc:407] Connection header received
0806 22:50:26.823321 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:26.823321 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:26.823332 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:26.823390 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:26.823769 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:26.824492 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:26.825227 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:26.825432 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:26.825494 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:26.828044 (+ 2550us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.218.193:40451: BlockingRecv error: recv got EOF from 127.28.218.193:40451 (error 108)
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":133,"threads_started":1}
Aug 06 22:50:26 dist-test-slave-1xrh krb5kdc[5743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.218.194@KRBTEST.COM for oryx/127.28.218.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:50:26 dist-test-slave-1xrh krb5kdc[5743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.218.194@KRBTEST.COM for oryx/127.28.218.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:50:26.948185 6825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:26.943689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:33979 (local address 127.28.218.194:58535)
0806 22:50:26.943840 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:26.943851 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:26.943925 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:26.944048 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:26.944049 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:26.944209 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:26.944256 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:26.944257 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:26.944860 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:26.944861 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:26.945485 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:26.945488 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:26.945835 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:26.945836 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:26.945876 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:26.946308 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:50:26.946313 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:26.948118 (+ 1805us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.218.254:33979: Server oryx/127.28.218.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":87,"threads_started":1}
W20260806 22:50:26.948313 6824 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:50:26.943741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.194:58535 (local address 127.28.218.254:33979)
0806 22:50:26.943816 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:26.943818 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:26.943909 (+ 91us) server_negotiation.cc:407] Connection header received
0806 22:50:26.943978 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:26.943979 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:26.943989 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:26.944044 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:26.944312 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:26.944797 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:26.945563 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:26.945757 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:26.945806 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:26.948210 (+ 2404us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.218.194:58535: BlockingRecv error: recv got EOF from 127.28.218.194:58535 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Aug 06 22:50:27 dist-test-slave-1xrh krb5kdc[5743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.218.195@KRBTEST.COM for oryx/127.28.218.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:50:27 dist-test-slave-1xrh krb5kdc[5743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.218.195@KRBTEST.COM for oryx/127.28.218.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:50:27.084214 6826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:27.079265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:33979 (local address 127.28.218.195:46559)
0806 22:50:27.079442 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:27.079454 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:27.079545 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:27.079702 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:27.079703 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:27.079889 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:27.079937 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:27.079938 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:27.080599 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:27.080601 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:27.081144 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:27.081146 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:27.081533 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:27.081534 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:27.081594 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:27.082113 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:50:27.082118 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:27.084119 (+ 2001us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.218.254:33979: Server oryx/127.28.218.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":89,"threads_started":1}
W20260806 22:50:27.084450 6824 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:50:27.079352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.195:46559 (local address 127.28.218.254:33979)
0806 22:50:27.079431 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:27.079433 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:27.079523 (+ 90us) server_negotiation.cc:407] Connection header received
0806 22:50:27.079603 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:27.079604 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:27.079619 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:27.079693 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:27.080030 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:27.080515 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:27.081233 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:27.081445 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:27.081533 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:27.084244 (+ 2711us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.218.195:46559: BlockingRecv error: recv got EOF from 127.28.218.195:46559 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 06 22:50:27 dist-test-slave-1xrh krb5kdc[5743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.218.193@KRBTEST.COM for oryx/127.28.218.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:50:27 dist-test-slave-1xrh krb5kdc[5743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.218.193@KRBTEST.COM for oryx/127.28.218.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:50:27.833259 6827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:27.828481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:33979 (local address 127.28.218.193:52733)
0806 22:50:27.828611 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:27.828624 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:27.828688 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:27.828852 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:27.828853 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:27.829025 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:27.829073 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:27.829074 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:27.829786 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:27.829787 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:27.830457 (+ 670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:27.830460 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:27.830930 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:27.830931 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:27.830977 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:27.831479 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:50:27.831483 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:27.833202 (+ 1719us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.218.254:33979: Server oryx/127.28.218.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":59,"threads_started":1}
W20260806 22:50:27.833379 6828 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:50:27.828558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.193:52733 (local address 127.28.218.254:33979)
0806 22:50:27.828697 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:27.828699 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:27.828727 (+ 28us) server_negotiation.cc:407] Connection header received
0806 22:50:27.828764 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:27.828764 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:27.828775 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:27.828831 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:27.829200 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:27.829706 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:27.830548 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:27.830859 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:27.830911 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:27.833295 (+ 2384us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.218.193:52733: BlockingRecv error: recv got EOF from 127.28.218.193:52733 (error 108)
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":68,"threads_started":1}
Aug 06 22:50:27 dist-test-slave-1xrh krb5kdc[5743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.218.194@KRBTEST.COM for oryx/127.28.218.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:50:27 dist-test-slave-1xrh krb5kdc[5743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.218.194@KRBTEST.COM for oryx/127.28.218.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:50:27.959630 6831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:27.948656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:33979 (local address 127.28.218.194:42149)
0806 22:50:27.948821 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:27.948832 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:27.948904 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:27.949044 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:27.949044 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:27.949201 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:27.949249 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:27.949250 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:27.949863 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:27.949864 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:27.950377 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:27.950378 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:27.950696 (+ 318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:27.950697 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:27.950736 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:27.957509 (+ 6773us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:50:27.957514 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:27.959544 (+ 2030us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.218.254:33979: Server oryx/127.28.218.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":90,"threads_started":1}
W20260806 22:50:27.959743 6828 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:50:27.948694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.194:42149 (local address 127.28.218.254:33979)
0806 22:50:27.948762 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:27.948764 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:27.948907 (+ 143us) server_negotiation.cc:407] Connection header received
0806 22:50:27.948973 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:27.948974 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:27.948986 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:27.949036 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:27.949306 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:27.949796 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:27.950448 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:27.950633 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:27.950685 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:27.959650 (+ 8965us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.218.194:42149: BlockingRecv error: recv got EOF from 127.28.218.194:42149 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 06 22:50:28 dist-test-slave-1xrh krb5kdc[5743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.218.195@KRBTEST.COM for oryx/127.28.218.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:50:28 dist-test-slave-1xrh krb5kdc[5743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.218.195@KRBTEST.COM for oryx/127.28.218.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:50:28.089850 6832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:28.085158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:33979 (local address 127.28.218.195:56307)
0806 22:50:28.085300 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:28.085310 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:28.085364 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:28.085493 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:28.085494 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:28.085652 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:28.085700 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:28.085701 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:28.086339 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:28.086340 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:28.086844 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:28.086846 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:28.087211 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:28.087212 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:28.087252 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:28.087824 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:50:28.087829 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:28.089757 (+ 1928us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.218.254:33979: Server oryx/127.28.218.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":85,"threads_started":1}
W20260806 22:50:28.089903 6828 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:50:28.085226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.195:56307 (local address 127.28.218.254:33979)
0806 22:50:28.085298 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:28.085300 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:28.085353 (+ 53us) server_negotiation.cc:407] Connection header received
0806 22:50:28.085419 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:28.085419 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:28.085432 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:28.085472 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:28.085762 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:28.086274 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:28.086922 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:28.087098 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:28.087213 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:28.089838 (+ 2625us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.218.195:56307: BlockingRecv error: recv got EOF from 127.28.218.195:56307 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 06 22:50:28 dist-test-slave-1xrh krb5kdc[5743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.218.193@KRBTEST.COM for oryx/127.28.218.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:50:28 dist-test-slave-1xrh krb5kdc[5743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.218.193@KRBTEST.COM for oryx/127.28.218.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:50:28.838137 6833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:28.833732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:33979 (local address 127.28.218.193:52175)
0806 22:50:28.833888 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:28.833898 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:28.833950 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:28.834075 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:28.834076 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:28.834256 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:28.834304 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:28.834306 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:28.834953 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:28.834954 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:28.835534 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:28.835536 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:28.835866 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:28.835866 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:28.835908 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:28.836348 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:50:28.836352 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:28.838078 (+ 1726us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.218.254:33979: Server oryx/127.28.218.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":93,"threads_started":1}
W20260806 22:50:28.838246 6834 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:50:28.833801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.193:52175 (local address 127.28.218.254:33979)
0806 22:50:28.833946 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:28.833948 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:28.833957 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:50:28.833995 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:28.833995 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:28.834006 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:28.834061 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:28.834367 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:28.834885 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:28.835604 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:28.835789 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:28.835849 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:28.838166 (+ 2317us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.218.193:52175: BlockingRecv error: recv got EOF from 127.28.218.193:52175 (error 108)
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":94,"threads_started":1}
Aug 06 22:50:28 dist-test-slave-1xrh krb5kdc[5743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.218.194@KRBTEST.COM for oryx/127.28.218.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:50:28 dist-test-slave-1xrh krb5kdc[5743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.218.194@KRBTEST.COM for oryx/127.28.218.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:50:28.964793 6835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:28.960120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:33979 (local address 127.28.218.194:37227)
0806 22:50:28.960321 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:28.960335 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:28.960406 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:28.960549 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:28.960549 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:28.960722 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:28.960770 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:28.960772 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:28.961448 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:28.961449 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:28.962006 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:28.962008 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:28.962407 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:28.962408 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:28.962459 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:28.962931 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:50:28.962935 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:28.964734 (+ 1799us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.218.254:33979: Server oryx/127.28.218.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":77,"threads_started":1}
W20260806 22:50:28.964913 6834 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:50:28.960294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.194:37227 (local address 127.28.218.254:33979)
0806 22:50:28.960408 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:28.960409 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:28.960444 (+ 35us) server_negotiation.cc:407] Connection header received
0806 22:50:28.960469 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:28.960470 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:28.960482 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:28.960524 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:28.960852 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:28.961347 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:28.962098 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:28.962307 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:28.962384 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:28.964829 (+ 2445us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.218.194:37227: BlockingRecv error: recv got EOF from 127.28.218.194:37227 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
Aug 06 22:50:29 dist-test-slave-1xrh krb5kdc[5743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.218.195@KRBTEST.COM for oryx/127.28.218.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:50:29 dist-test-slave-1xrh krb5kdc[5743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.218.195@KRBTEST.COM for oryx/127.28.218.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:50:29.095232 6836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:29.090405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:33979 (local address 127.28.218.195:60969)
0806 22:50:29.090602 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:29.090615 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:29.090718 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:29.090864 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:29.090865 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:29.091059 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:29.091106 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:29.091108 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:29.091784 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:29.091786 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:29.092321 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:29.092323 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:29.092710 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:29.092711 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:29.092773 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:29.093290 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:50:29.093295 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:29.095167 (+ 1872us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.218.254:33979: Server oryx/127.28.218.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":123,"threads_started":1}
W20260806 22:50:29.095290 6834 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:50:29.090510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.195:60969 (local address 127.28.218.254:33979)
0806 22:50:29.090584 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:29.090586 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:29.090687 (+ 101us) server_negotiation.cc:407] Connection header received
0806 22:50:29.090773 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:29.090774 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:29.090795 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:29.090856 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:29.091215 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:29.091697 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:29.092434 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:29.092624 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:29.092712 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:29.095242 (+ 2530us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.218.195:60969: BlockingRecv error: recv got EOF from 127.28.218.195:60969 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 06 22:50:29 dist-test-slave-1xrh krb5kdc[5743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.218.193@KRBTEST.COM for oryx/127.28.218.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:50:29 dist-test-slave-1xrh krb5kdc[5743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.218.193@KRBTEST.COM for oryx/127.28.218.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:50:29.843377 6837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:29.838607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:33979 (local address 127.28.218.193:53053)
0806 22:50:29.838780 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:29.838790 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:29.838873 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:29.839075 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:29.839075 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:29.839292 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:29.839341 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:29.839343 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:29.839958 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:29.839959 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:29.840458 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:29.840460 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:29.840768 (+ 308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:29.840769 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:29.840807 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:29.841352 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:50:29.841357 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:29.843317 (+ 1960us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.218.254:33979: Server oryx/127.28.218.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":76,"threads_started":1}
W20260806 22:50:29.843482 6838 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:50:29.838684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.193:53053 (local address 127.28.218.254:33979)
0806 22:50:29.838810 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:29.838812 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:29.838871 (+ 59us) server_negotiation.cc:407] Connection header received
0806 22:50:29.838981 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:29.838982 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:29.838996 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:29.839068 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:29.839406 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:29.839891 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:29.840530 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:29.840707 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:29.840753 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:29.843404 (+ 2651us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.218.193:53053: BlockingRecv error: recv got EOF from 127.28.218.193:53053 (error 108)
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":69,"threads_started":1}
Aug 06 22:50:29 dist-test-slave-1xrh krb5kdc[5743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.218.194@KRBTEST.COM for oryx/127.28.218.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:50:29 dist-test-slave-1xrh krb5kdc[5743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.218.194@KRBTEST.COM for oryx/127.28.218.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:50:29.970031 6839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:29.965350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:33979 (local address 127.28.218.194:59835)
0806 22:50:29.965524 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:29.965535 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:29.965637 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:29.965817 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:29.965818 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:29.966044 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:29.966117 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:29.966119 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:29.966871 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:29.966872 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:29.967415 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:29.967416 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:29.967730 (+ 314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:29.967730 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:29.967782 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:29.968203 (+ 421us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:50:29.968207 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:29.969962 (+ 1755us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.218.254:33979: Server oryx/127.28.218.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":98,"threads_started":1}
W20260806 22:50:29.970098 6838 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:50:29.965402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.194:59835 (local address 127.28.218.254:33979)
0806 22:50:29.965474 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:29.965477 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:29.965624 (+ 147us) server_negotiation.cc:407] Connection header received
0806 22:50:29.965704 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:29.965705 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:29.965718 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:29.965787 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:29.966212 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:29.966798 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:29.967484 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:29.967667 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:29.967719 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:29.970051 (+ 2332us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.218.194:59835: BlockingRecv error: recv got EOF from 127.28.218.194:59835 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 06 22:50:30 dist-test-slave-1xrh krb5kdc[5743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.218.195@KRBTEST.COM for oryx/127.28.218.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:50:30 dist-test-slave-1xrh krb5kdc[5743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.218.195@KRBTEST.COM for oryx/127.28.218.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:50:30.100363 6840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:30.095790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:33979 (local address 127.28.218.195:41409)
0806 22:50:30.095928 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:30.095939 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:30.096001 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:30.096129 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:30.096130 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:30.096281 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:30.096327 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:30.096329 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:30.096933 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:30.096933 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:30.097477 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:30.097479 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:30.097912 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:30.097913 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:30.097951 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:30.098402 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:50:30.098406 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:30.100264 (+ 1858us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.218.254:33979: Server oryx/127.28.218.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":83,"threads_started":1}
W20260806 22:50:30.100422 6838 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:50:30.095866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.195:41409 (local address 127.28.218.254:33979)
0806 22:50:30.095956 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:30.095958 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:30.095986 (+ 28us) server_negotiation.cc:407] Connection header received
0806 22:50:30.096055 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:30.096056 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:30.096068 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:30.096108 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:30.096386 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:30.096866 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:30.097575 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:30.097796 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:30.097913 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:30.100341 (+ 2428us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.218.195:41409: BlockingRecv error: recv got EOF from 127.28.218.195:41409 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 06 22:50:30 dist-test-slave-1xrh krb5kdc[5743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.218.193@KRBTEST.COM for oryx/127.28.218.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:50:30 dist-test-slave-1xrh krb5kdc[5743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.218.193@KRBTEST.COM for oryx/127.28.218.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:50:30.848565 6841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:30.843908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:33979 (local address 127.28.218.193:60219)
0806 22:50:30.844050 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:30.844062 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:30.844112 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:30.844241 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:30.844241 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:30.844402 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:30.844452 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:30.844454 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:30.845270 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:30.845272 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:30.845877 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:30.845879 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:30.846197 (+ 318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:30.846198 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:30.846239 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:30.846736 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:50:30.846741 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:30.848509 (+ 1768us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.218.254:33979: Server oryx/127.28.218.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":76,"threads_started":1}
W20260806 22:50:30.848701 6842 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:50:30.843946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.193:60219 (local address 127.28.218.254:33979)
0806 22:50:30.844096 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:30.844098 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:30.844115 (+ 17us) server_negotiation.cc:407] Connection header received
0806 22:50:30.844156 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:30.844157 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:30.844166 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:30.844223 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:30.844513 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:30.845188 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:30.845949 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:30.846135 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:30.846186 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:30.848609 (+ 2423us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.218.193:60219: BlockingRecv error: recv got EOF from 127.28.218.193:60219 (error 108)
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":99,"threads_started":1}
Aug 06 22:50:30 dist-test-slave-1xrh krb5kdc[5743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.218.194@KRBTEST.COM for oryx/127.28.218.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:50:30 dist-test-slave-1xrh krb5kdc[5743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.218.194@KRBTEST.COM for oryx/127.28.218.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:50:30.975263 6843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:30.970538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:33979 (local address 127.28.218.194:57647)
0806 22:50:30.970741 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:30.970751 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:30.970811 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:30.970936 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:30.970936 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:30.971100 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:30.971212 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:30.971214 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:30.971860 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:30.971861 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:30.972365 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:30.972366 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:30.972702 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:30.972702 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:30.972741 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:30.973172 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:50:30.973176 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:30.975181 (+ 2005us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.218.254:33979: Server oryx/127.28.218.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":133,"threads_started":1}
W20260806 22:50:30.975378 6842 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:50:30.970615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.194:57647 (local address 127.28.218.254:33979)
0806 22:50:30.970702 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:30.970704 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:30.970814 (+ 110us) server_negotiation.cc:407] Connection header received
0806 22:50:30.970868 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:30.970869 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:30.970880 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:30.970926 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:30.971273 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:30.971775 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:30.972451 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:30.972641 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:30.972687 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:30.975298 (+ 2611us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.218.194:57647: BlockingRecv error: recv got EOF from 127.28.218.194:57647 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 06 22:50:31 dist-test-slave-1xrh krb5kdc[5743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.218.195@KRBTEST.COM for oryx/127.28.218.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:50:31 dist-test-slave-1xrh krb5kdc[5743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.218.195@KRBTEST.COM for oryx/127.28.218.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:50:31.105789 6844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:31.100877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:33979 (local address 127.28.218.195:33789)
0806 22:50:31.101055 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:31.101066 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:31.101125 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:31.101249 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:31.101250 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:31.101408 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:31.101458 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:31.101460 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:31.102097 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:31.102098 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:31.102651 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:31.102654 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:31.103065 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:31.103065 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:31.103105 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:31.103735 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:50:31.103741 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:31.105726 (+ 1985us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.218.254:33979: Server oryx/127.28.218.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":91,"threads_started":1}
W20260806 22:50:31.105945 6842 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:50:31.100935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.195:33789 (local address 127.28.218.254:33979)
0806 22:50:31.101005 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:31.101007 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:31.101124 (+ 117us) server_negotiation.cc:407] Connection header received
0806 22:50:31.101167 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:31.101168 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:31.101180 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:31.101228 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:31.101524 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:31.102032 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:31.102767 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:31.102995 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:31.103057 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:31.105835 (+ 2778us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.218.195:33789: BlockingRecv error: recv got EOF from 127.28.218.195:33789 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 06 22:50:31 dist-test-slave-1xrh krb5kdc[5743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.218.193@KRBTEST.COM for oryx/127.28.218.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:50:31 dist-test-slave-1xrh krb5kdc[5743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.218.193@KRBTEST.COM for oryx/127.28.218.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:50:31.853855 6845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:31.849040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:33979 (local address 127.28.218.193:55119)
0806 22:50:31.849193 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:31.849211 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:31.849259 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:31.849393 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:31.849393 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:31.849629 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:31.849695 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:31.849698 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:31.850831 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:31.850832 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:31.851399 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:31.851401 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:31.851723 (+ 322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:31.851724 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:31.851761 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:31.852162 (+ 401us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:50:31.852166 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:31.853801 (+ 1635us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.218.254:33979: Server oryx/127.28.218.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":96,"threads_started":1}
W20260806 22:50:31.854007 6846 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:50:31.849152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.193:55119 (local address 127.28.218.254:33979)
0806 22:50:31.849272 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:31.849274 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:31.849287 (+ 13us) server_negotiation.cc:407] Connection header received
0806 22:50:31.849309 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:31.849309 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:31.849318 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:31.849375 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:31.849796 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:31.850743 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:31.851469 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:31.851663 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:31.851711 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:31.853923 (+ 2212us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.218.193:55119: BlockingRecv error: recv got EOF from 127.28.218.193:55119 (error 108)
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":71,"threads_started":1}
Aug 06 22:50:31 dist-test-slave-1xrh krb5kdc[5743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.218.194@KRBTEST.COM for oryx/127.28.218.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:50:31 dist-test-slave-1xrh krb5kdc[5743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.218.194@KRBTEST.COM for oryx/127.28.218.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:50:31.980655 6847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:31.975849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:33979 (local address 127.28.218.194:49057)
0806 22:50:31.976030 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:31.976042 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:31.976132 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:31.976283 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:31.976283 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:31.976487 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:31.976536 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:31.976538 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:31.977353 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:31.977355 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:31.977885 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:31.977887 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:31.978248 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:31.978249 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:31.978306 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:31.978759 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:50:31.978764 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:31.980598 (+ 1834us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.218.254:33979: Server oryx/127.28.218.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":97,"threads_started":1}
W20260806 22:50:31.980780 6846 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:50:31.975940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.194:49057 (local address 127.28.218.254:33979)
0806 22:50:31.976011 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:31.976013 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:31.976113 (+ 100us) server_negotiation.cc:407] Connection header received
0806 22:50:31.976189 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:31.976190 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:31.976204 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:31.976276 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:31.976621 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:31.977263 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:31.977975 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:31.978174 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:31.978251 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:31.980700 (+ 2449us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.218.194:49057: BlockingRecv error: recv got EOF from 127.28.218.194:49057 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 06 22:50:32 dist-test-slave-1xrh krb5kdc[5743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.218.195@KRBTEST.COM for oryx/127.28.218.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:50:32 dist-test-slave-1xrh krb5kdc[5743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.218.195@KRBTEST.COM for oryx/127.28.218.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:50:32.110867 6848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:32.106209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:33979 (local address 127.28.218.195:39419)
0806 22:50:32.106309 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:32.106320 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:32.106384 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:32.106506 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:32.106506 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:32.106655 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:32.106703 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:32.106705 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:32.107436 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:32.107437 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:32.107977 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:32.107979 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:32.108368 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:32.108369 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:32.108408 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:32.108854 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:50:32.108873 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:32.110785 (+ 1912us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.218.254:33979: Server oryx/127.28.218.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":82,"thread_start_us":54,"threads_started":1}
W20260806 22:50:32.110920 6846 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:50:32.106264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.195:39419 (local address 127.28.218.254:33979)
0806 22:50:32.106307 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:32.106309 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:32.106368 (+ 59us) server_negotiation.cc:407] Connection header received
0806 22:50:32.106434 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:32.106435 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:32.106448 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:32.106500 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:32.106759 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:32.107331 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:32.108069 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:32.108261 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:32.108356 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:32.110873 (+ 2517us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.218.195:39419: BlockingRecv error: recv got EOF from 127.28.218.195:39419 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
Aug 06 22:50:32 dist-test-slave-1xrh krb5kdc[5743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.218.193@KRBTEST.COM for oryx/127.28.218.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:50:32 dist-test-slave-1xrh krb5kdc[5743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.218.193@KRBTEST.COM for oryx/127.28.218.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:50:32.859056 6849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:32.854357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:33979 (local address 127.28.218.193:38511)
0806 22:50:32.854508 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:32.854518 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:32.854566 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:32.854700 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:32.854701 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:32.854862 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:32.854929 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:32.854931 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:32.855592 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:32.855593 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:32.856116 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:32.856117 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:32.856440 (+ 323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:32.856441 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:32.856477 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:32.856969 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:50:32.856973 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:32.858937 (+ 1964us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.218.254:33979: Server oryx/127.28.218.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":89,"threads_started":1}
W20260806 22:50:32.859090 6850 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:50:32.854418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.193:38511 (local address 127.28.218.254:33979)
0806 22:50:32.854564 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:32.854566 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:32.854574 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:50:32.854603 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:32.854604 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:32.854623 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:32.854681 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:32.854992 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:32.855528 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:32.856184 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:32.856379 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:32.856425 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:32.859016 (+ 2591us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.218.193:38511: BlockingRecv error: recv got EOF from 127.28.218.193:38511 (error 108)
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":112,"threads_started":1}
Aug 06 22:50:32 dist-test-slave-1xrh krb5kdc[5743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.218.194@KRBTEST.COM for oryx/127.28.218.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:50:32 dist-test-slave-1xrh krb5kdc[5743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.218.194@KRBTEST.COM for oryx/127.28.218.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:50:32.995676 6851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:32.981152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:33979 (local address 127.28.218.194:60023)
0806 22:50:32.981281 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:32.981301 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:32.981364 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:32.981511 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:32.981512 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:32.981679 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:32.981728 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:32.981730 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:32.982422 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:32.982423 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:32.982968 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:32.982969 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:32.983386 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:32.983387 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:32.983444 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:32.993697 (+ 10253us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:50:32.993701 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:32.995618 (+ 1917us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.218.254:33979: Server oryx/127.28.218.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":57,"threads_started":1}
W20260806 22:50:32.995793 6850 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:50:32.981246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.194:60023 (local address 127.28.218.254:33979)
0806 22:50:32.981322 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:32.981324 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:32.981348 (+ 24us) server_negotiation.cc:407] Connection header received
0806 22:50:32.981409 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:32.981410 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:32.981433 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:32.981490 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:32.981820 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:32.982322 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:32.983058 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:32.983312 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:32.983388 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:32.995717 (+ 12329us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.218.194:60023: BlockingRecv error: recv got EOF from 127.28.218.194:60023 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 06 22:50:33 dist-test-slave-1xrh krb5kdc[5743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.218.195@KRBTEST.COM for oryx/127.28.218.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:50:33 dist-test-slave-1xrh krb5kdc[5743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.218.195@KRBTEST.COM for oryx/127.28.218.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:50:33.115873 6852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:33.111349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:33979 (local address 127.28.218.195:49177)
0806 22:50:33.111510 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:33.111523 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:33.111585 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:33.111706 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:33.111707 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:33.111880 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:33.111928 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:33.111930 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:33.112518 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:33.112519 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:33.113050 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:33.113052 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:33.113351 (+ 299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:33.113352 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:33.113390 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:33.113828 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:50:33.113832 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:33.115733 (+ 1901us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.218.254:33979: Server oryx/127.28.218.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":86,"threads_started":1}
W20260806 22:50:33.115913 6850 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:50:33.111396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.195:49177 (local address 127.28.218.254:33979)
0806 22:50:33.111465 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:33.111467 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:33.111580 (+ 113us) server_negotiation.cc:407] Connection header received
0806 22:50:33.111627 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:33.111628 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:33.111640 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:33.111687 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:33.111982 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:33.112456 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:33.113113 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:33.113291 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:33.113339 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:33.115865 (+ 2526us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.218.195:49177: BlockingRecv error: recv got EOF from 127.28.218.195:49177 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Aug 06 22:50:33 dist-test-slave-1xrh krb5kdc[5743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.218.193@KRBTEST.COM for oryx/127.28.218.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:50:33 dist-test-slave-1xrh krb5kdc[5743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.218.193@KRBTEST.COM for oryx/127.28.218.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:50:33.863884 6853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:33.859487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:33979 (local address 127.28.218.193:52923)
0806 22:50:33.859634 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:33.859644 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:33.859701 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:33.859855 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:33.859856 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:33.860013 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:33.860063 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:33.860064 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:33.860691 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:33.860692 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:33.861200 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:33.861201 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:33.861532 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:33.861532 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:33.861571 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:33.862002 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:50:33.862006 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:33.863797 (+ 1791us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.218.254:33979: Server oryx/127.28.218.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":98,"threads_started":1}
W20260806 22:50:33.863989 6854 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:50:33.859554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.193:52923 (local address 127.28.218.254:33979)
0806 22:50:33.859695 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:33.859697 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:33.859706 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:50:33.859769 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:33.859770 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:33.859781 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:33.859840 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:33.860127 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:33.860626 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:33.861270 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:33.861459 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:33.861508 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:33.863913 (+ 2405us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.218.193:52923: BlockingRecv error: recv got EOF from 127.28.218.193:52923 (error 108)
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":92,"threads_started":1}
Aug 06 22:50:33 dist-test-slave-1xrh krb5kdc[5743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.218.194@KRBTEST.COM for oryx/127.28.218.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:50:33 dist-test-slave-1xrh krb5kdc[5743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.218.194@KRBTEST.COM for oryx/127.28.218.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:50:34.001334 6855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:33.996259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:33979 (local address 127.28.218.194:47207)
0806 22:50:33.996483 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:33.996497 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:33.996570 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:33.996716 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:33.996716 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:33.996929 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:33.996982 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:33.996984 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:33.997751 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:33.997752 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:33.998288 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:33.998290 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:33.998685 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:33.998686 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:33.998751 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:33.999339 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:50:33.999343 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:34.001260 (+ 1917us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.218.254:33979: Server oryx/127.28.218.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":153,"threads_started":1}
W20260806 22:50:34.001447 6854 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:50:33.996323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.194:47207 (local address 127.28.218.254:33979)
0806 22:50:33.996393 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:33.996395 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:33.996567 (+ 172us) server_negotiation.cc:407] Connection header received
0806 22:50:33.996628 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:33.996629 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:33.996639 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:33.996694 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:33.997067 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:33.997664 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:33.998387 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:33.998596 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:33.998685 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:34.001353 (+ 2668us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.218.194:47207: BlockingRecv error: recv got EOF from 127.28.218.194:47207 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Aug 06 22:50:34 dist-test-slave-1xrh krb5kdc[5743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.218.195@KRBTEST.COM for oryx/127.28.218.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:50:34 dist-test-slave-1xrh krb5kdc[5743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.218.195@KRBTEST.COM for oryx/127.28.218.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:50:34.121129 6856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:34.116336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:33979 (local address 127.28.218.195:41663)
0806 22:50:34.116520 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:34.116533 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:34.116597 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:34.116722 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:34.116723 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:34.116903 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:34.116971 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:34.116972 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:34.117668 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:34.117669 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:34.118285 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:34.118287 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:34.118663 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:34.118664 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:34.118736 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:34.119261 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:50:34.119267 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:34.121036 (+ 1769us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.218.254:33979: Server oryx/127.28.218.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":98,"threads_started":1}
W20260806 22:50:34.121170 6854 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:50:34.116429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.195:41663 (local address 127.28.218.254:33979)
0806 22:50:34.116523 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:34.116526 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:34.116582 (+ 56us) server_negotiation.cc:407] Connection header received
0806 22:50:34.116650 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:34.116651 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:34.116665 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:34.116717 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:34.117070 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:34.117585 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:34.118374 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:34.118588 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:34.118665 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:34.121107 (+ 2442us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.218.195:41663: BlockingRecv error: recv got EOF from 127.28.218.195:41663 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 06 22:50:34 dist-test-slave-1xrh krb5kdc[5743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.218.193@KRBTEST.COM for oryx/127.28.218.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:50:34 dist-test-slave-1xrh krb5kdc[5743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.218.193@KRBTEST.COM for oryx/127.28.218.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:50:34.869174 6857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:34.864361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:33979 (local address 127.28.218.193:51305)
0806 22:50:34.864516 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:34.864527 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:34.864580 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:34.864709 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:34.864710 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:34.864873 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:34.864933 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:34.864935 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:34.865628 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:34.865634 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:34.866218 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:34.866220 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:34.866675 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:34.866676 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:34.866736 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:34.867307 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:50:34.867312 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:34.869102 (+ 1790us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.218.254:33979: Server oryx/127.28.218.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":88,"threads_started":1}
W20260806 22:50:34.869293 6858 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:50:34.864449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.193:51305 (local address 127.28.218.254:33979)
0806 22:50:34.864582 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:34.864584 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:34.864610 (+ 26us) server_negotiation.cc:407] Connection header received
0806 22:50:34.864630 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:34.864631 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:34.864641 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:34.864688 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:34.865004 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:34.865549 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:34.866302 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:34.866589 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:34.866673 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:34.869202 (+ 2529us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.218.193:51305: BlockingRecv error: recv got EOF from 127.28.218.193:51305 (error 108)
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":80,"threads_started":1}
Aug 06 22:50:35 dist-test-slave-1xrh krb5kdc[5743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.218.194@KRBTEST.COM for oryx/127.28.218.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:50:35 dist-test-slave-1xrh krb5kdc[5743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.218.194@KRBTEST.COM for oryx/127.28.218.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:50:35.007038 6859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:35.001878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:33979 (local address 127.28.218.194:35665)
0806 22:50:35.002088 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:35.002101 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:35.002188 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:35.002346 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:35.002346 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:35.002542 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:35.002588 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:35.002590 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:35.003462 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:35.003463 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:35.004068 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:35.004070 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:35.004466 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:35.004467 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:35.004532 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:35.005077 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:50:35.005081 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:35.006964 (+ 1883us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.218.254:33979: Server oryx/127.28.218.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":140,"threads_started":1}
W20260806 22:50:35.007107 6858 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:50:35.001976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.194:35665 (local address 127.28.218.254:33979)
0806 22:50:35.002072 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:35.002074 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:35.002170 (+ 96us) server_negotiation.cc:407] Connection header received
0806 22:50:35.002267 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:35.002267 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:35.002280 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:35.002339 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:35.002667 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:35.003369 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:35.004163 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:35.004382 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:35.004455 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:35.007057 (+ 2602us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.218.194:35665: BlockingRecv error: recv got EOF from 127.28.218.194:35665 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 06 22:50:35 dist-test-slave-1xrh krb5kdc[5743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.218.195@KRBTEST.COM for oryx/127.28.218.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:50:35 dist-test-slave-1xrh krb5kdc[5743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.218.195@KRBTEST.COM for oryx/127.28.218.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:50:35.126760 6860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:35.121648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:33979 (local address 127.28.218.195:32929)
0806 22:50:35.121829 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:35.121842 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:35.121919 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:35.122118 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:35.122119 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:35.122322 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:35.122370 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:35.122372 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:35.123073 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:35.123074 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:35.123654 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:35.123656 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:35.124047 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:35.124048 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:35.124114 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:35.124616 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:50:35.124621 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:35.126683 (+ 2062us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.218.254:33979: Server oryx/127.28.218.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":87,"threads_started":1}
W20260806 22:50:35.126817 6858 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:50:35.121738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.195:32929 (local address 127.28.218.254:33979)
0806 22:50:35.121870 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:35.121872 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:35.121898 (+ 26us) server_negotiation.cc:407] Connection header received
0806 22:50:35.122007 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:35.122008 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:35.122044 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:35.122106 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:35.122456 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:35.122971 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:35.123763 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:35.123972 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:35.124049 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:35.126768 (+ 2719us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.218.195:32929: BlockingRecv error: recv got EOF from 127.28.218.195:32929 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 06 22:50:35 dist-test-slave-1xrh krb5kdc[5743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.218.193@KRBTEST.COM for oryx/127.28.218.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:50:35 dist-test-slave-1xrh krb5kdc[5743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.218.193@KRBTEST.COM for oryx/127.28.218.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:50:35.874188 6861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:35.869679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:33979 (local address 127.28.218.193:34031)
0806 22:50:35.869828 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:35.869842 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:35.869905 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:35.870041 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:35.870042 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:35.870206 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:35.870256 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:35.870258 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:35.870911 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:35.870912 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:35.871481 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:35.871483 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:35.871821 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:35.871822 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:35.871860 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:35.872298 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:50:35.872302 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:35.874121 (+ 1819us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.218.254:33979: Server oryx/127.28.218.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":69,"threads_started":1}
W20260806 22:50:35.874264 6862 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:50:35.869746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.193:34031 (local address 127.28.218.254:33979)
0806 22:50:35.869892 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:35.869894 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:35.869902 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:50:35.869962 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:35.869963 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:35.869973 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:35.870025 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:35.870323 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:35.870843 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:35.871564 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:35.871756 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:35.871809 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:35.874210 (+ 2401us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.218.193:34031: BlockingRecv error: recv got EOF from 127.28.218.193:34031 (error 108)
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":69,"threads_started":1}
Aug 06 22:50:36 dist-test-slave-1xrh krb5kdc[5743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.218.194@KRBTEST.COM for oryx/127.28.218.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:50:36 dist-test-slave-1xrh krb5kdc[5743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.218.194@KRBTEST.COM for oryx/127.28.218.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:50:36.012292 6863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:36.007602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:33979 (local address 127.28.218.194:49403)
0806 22:50:36.007776 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:36.007787 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:36.007889 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:36.008050 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:36.008051 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:36.008216 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:36.008265 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:36.008266 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:36.008909 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:36.008910 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:36.009411 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:36.009412 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:36.009807 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:36.009808 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:36.009845 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:36.010330 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:50:36.010334 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:36.012227 (+ 1893us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.218.254:33979: Server oryx/127.28.218.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":92,"threads_started":1}
W20260806 22:50:36.012365 6862 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:50:36.007711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.194:49403 (local address 127.28.218.254:33979)
0806 22:50:36.007801 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:36.007803 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:36.007869 (+ 66us) server_negotiation.cc:407] Connection header received
0806 22:50:36.007949 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:36.007949 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:36.007961 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:36.008022 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:36.008349 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:36.008847 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:36.009492 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:36.009742 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:36.009797 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:36.012316 (+ 2519us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.218.194:49403: BlockingRecv error: recv got EOF from 127.28.218.194:49403 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 06 22:50:36 dist-test-slave-1xrh krb5kdc[5743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.218.195@KRBTEST.COM for oryx/127.28.218.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:50:36 dist-test-slave-1xrh krb5kdc[5743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.218.195@KRBTEST.COM for oryx/127.28.218.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:50:36.132377 6864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:36.127397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:33979 (local address 127.28.218.195:60301)
0806 22:50:36.127540 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:36.127551 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:36.127645 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:36.127826 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:36.127827 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:36.128083 (+ 256us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:36.128140 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:36.128142 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:36.128960 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:36.128961 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:36.129517 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:36.129519 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:36.129901 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:36.129902 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:36.129988 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:36.130469 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:50:36.130474 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:36.132286 (+ 1812us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.218.254:33979: Server oryx/127.28.218.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":72,"threads_started":1}
W20260806 22:50:36.132421 6862 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:50:36.127460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.195:60301 (local address 127.28.218.254:33979)
0806 22:50:36.127513 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:36.127515 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:36.127641 (+ 126us) server_negotiation.cc:407] Connection header received
0806 22:50:36.127725 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:36.127726 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:36.127739 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:36.127804 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:36.128208 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:36.128885 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:36.129604 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:36.129832 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:36.129904 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:36.132358 (+ 2454us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.218.195:60301: BlockingRecv error: recv got EOF from 127.28.218.195:60301 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
Aug 06 22:50:36 dist-test-slave-1xrh krb5kdc[5743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.218.193@KRBTEST.COM for oryx/127.28.218.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:50:36 dist-test-slave-1xrh krb5kdc[5743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.218.193@KRBTEST.COM for oryx/127.28.218.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:50:36.879148 6865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:36.874690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:33979 (local address 127.28.218.193:42065)
0806 22:50:36.874845 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:36.874855 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:36.874905 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:36.875041 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:36.875042 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:36.875228 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:36.875285 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:36.875286 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:36.875928 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:36.875929 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:36.876459 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:36.876460 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:36.876777 (+ 317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:36.876778 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:36.876823 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:36.877332 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:50:36.877337 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:36.879071 (+ 1734us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.218.254:33979: Server oryx/127.28.218.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":89,"threads_started":1}
W20260806 22:50:36.879313 6866 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:50:36.874765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.193:42065 (local address 127.28.218.254:33979)
0806 22:50:36.874904 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:36.874907 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:36.874940 (+ 33us) server_negotiation.cc:407] Connection header received
0806 22:50:36.874960 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:36.874961 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:36.874971 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:36.875021 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:36.875353 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:36.875852 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:36.876533 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:36.876714 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:36.876762 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:36.879221 (+ 2459us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.218.193:42065: BlockingRecv error: recv got EOF from 127.28.218.193:42065 (error 108)
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":79,"threads_started":1}
Aug 06 22:50:37 dist-test-slave-1xrh krb5kdc[5743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.218.194@KRBTEST.COM for oryx/127.28.218.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:50:37 dist-test-slave-1xrh krb5kdc[5743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.218.194@KRBTEST.COM for oryx/127.28.218.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:50:37.017561 6867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:37.012853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:33979 (local address 127.28.218.194:45603)
0806 22:50:37.013001 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:37.013020 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:37.013102 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:37.013254 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:37.013255 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:37.013453 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:37.013498 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:37.013500 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:37.014165 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:37.014167 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:37.014747 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:37.014749 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:37.015119 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:37.015158 (+ 39us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:37.015221 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:37.015657 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:50:37.015661 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:37.017495 (+ 1834us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.218.254:33979: Server oryx/127.28.218.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":64,"threads_started":1}
W20260806 22:50:37.017688 6866 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:50:37.013019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.194:45603 (local address 127.28.218.254:33979)
0806 22:50:37.013085 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:37.013087 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:37.013097 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:50:37.013162 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:37.013163 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:37.013176 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:37.013246 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:37.013586 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:37.014076 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:37.014836 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:37.015042 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:37.015120 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:37.017586 (+ 2466us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.218.194:45603: BlockingRecv error: recv got EOF from 127.28.218.194:45603 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 06 22:50:37 dist-test-slave-1xrh krb5kdc[5743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.218.195@KRBTEST.COM for oryx/127.28.218.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:50:37 dist-test-slave-1xrh krb5kdc[5743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.218.195@KRBTEST.COM for oryx/127.28.218.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:50:37.137398 6868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:37.132840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:33979 (local address 127.28.218.195:40587)
0806 22:50:37.133019 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:37.133029 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:37.133087 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:37.133204 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:37.133204 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:37.133366 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:37.133415 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:37.133416 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:37.134057 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:37.134057 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:37.134578 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:37.134579 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:37.134875 (+ 296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:37.134876 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:37.134912 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:37.135461 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:50:37.135466 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:37.137298 (+ 1832us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.218.254:33979: Server oryx/127.28.218.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":119,"threads_started":1}
W20260806 22:50:37.137451 6866 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:50:37.132888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.195:40587 (local address 127.28.218.254:33979)
0806 22:50:37.132966 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:37.132969 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:37.133084 (+ 115us) server_negotiation.cc:407] Connection header received
0806 22:50:37.133128 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:37.133129 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:37.133141 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:37.133186 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:37.133476 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:37.133993 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:37.134643 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:37.134816 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:37.134863 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:37.137402 (+ 2539us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.218.195:40587: BlockingRecv error: recv got EOF from 127.28.218.195:40587 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 06 22:50:37 dist-test-slave-1xrh krb5kdc[5743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.218.193@KRBTEST.COM for oryx/127.28.218.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:50:37 dist-test-slave-1xrh krb5kdc[5743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.218.193@KRBTEST.COM for oryx/127.28.218.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:50:37.884384 6869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:37.879756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:33979 (local address 127.28.218.193:57053)
0806 22:50:37.879956 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:37.879969 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:37.880042 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:37.880197 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:37.880198 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:37.880445 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:37.880504 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:37.880506 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:37.881154 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:37.881155 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:37.881694 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:37.881696 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:37.882012 (+ 316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:37.882013 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:37.882050 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:37.882483 (+ 433us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:50:37.882487 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:37.884326 (+ 1839us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.218.254:33979: Server oryx/127.28.218.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":90,"threads_started":1}
W20260806 22:50:37.884547 6870 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:50:37.879810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.193:57053 (local address 127.28.218.254:33979)
0806 22:50:37.879940 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:37.879943 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:37.880034 (+ 91us) server_negotiation.cc:407] Connection header received
0806 22:50:37.880088 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:37.880089 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:37.880103 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:37.880172 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:37.880585 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:37.881083 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:37.881774 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:37.881952 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:37.881998 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:37.884443 (+ 2445us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.218.193:57053: BlockingRecv error: recv got EOF from 127.28.218.193:57053 (error 108)
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":52,"threads_started":1}
Aug 06 22:50:38 dist-test-slave-1xrh krb5kdc[5743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.218.194@KRBTEST.COM for oryx/127.28.218.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:50:38 dist-test-slave-1xrh krb5kdc[5743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.218.194@KRBTEST.COM for oryx/127.28.218.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:50:38.029488 6871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:38.018035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:33979 (local address 127.28.218.194:35197)
0806 22:50:38.018202 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:38.018215 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:38.018294 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:38.018456 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:38.018456 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:38.018654 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:38.018702 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:38.018705 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:38.019626 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:38.019627 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:38.020154 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:38.020156 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:38.020549 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:38.020550 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:38.020608 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:38.027146 (+ 6538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:50:38.027152 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:38.029245 (+ 2093us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.218.254:33979: Server oryx/127.28.218.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":88,"threads_started":1}
W20260806 22:50:38.029604 6870 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:50:38.018121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.194:35197 (local address 127.28.218.254:33979)
0806 22:50:38.018205 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:38.018208 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:38.018284 (+ 76us) server_negotiation.cc:407] Connection header received
0806 22:50:38.018381 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:38.018382 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:38.018395 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:38.018450 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:38.018792 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:38.019549 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:38.020244 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:38.020472 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:38.020551 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:38.029409 (+ 8858us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.218.194:35197: BlockingRecv error: recv got EOF from 127.28.218.194:35197 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 06 22:50:38 dist-test-slave-1xrh krb5kdc[5743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.218.195@KRBTEST.COM for oryx/127.28.218.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:50:38 dist-test-slave-1xrh krb5kdc[5743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.218.195@KRBTEST.COM for oryx/127.28.218.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:50:38.142333 6872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:38.137896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:33979 (local address 127.28.218.195:60641)
0806 22:50:38.138079 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:38.138092 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:38.138176 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:38.138335 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:38.138336 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:38.138523 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:38.138571 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:38.138573 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:38.139226 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:38.139227 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:38.139718 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:38.139719 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:38.140034 (+ 315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:38.140034 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:38.140078 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:38.140534 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:50:38.140538 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:38.142276 (+ 1738us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.218.254:33979: Server oryx/127.28.218.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":114,"threads_started":1}
W20260806 22:50:38.142493 6870 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:50:38.137968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.195:60641 (local address 127.28.218.254:33979)
0806 22:50:38.138021 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:38.138023 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:38.138167 (+ 144us) server_negotiation.cc:407] Connection header received
0806 22:50:38.138233 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:38.138233 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:38.138247 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:38.138311 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:38.138643 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:38.139164 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:38.139789 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:38.139961 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:38.140008 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:38.142398 (+ 2390us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.218.195:60641: BlockingRecv error: recv got EOF from 127.28.218.195:60641 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
Aug 06 22:50:38 dist-test-slave-1xrh krb5kdc[5743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.218.193@KRBTEST.COM for oryx/127.28.218.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:50:38 dist-test-slave-1xrh krb5kdc[5743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.218.193@KRBTEST.COM for oryx/127.28.218.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:50:38.889580 6873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:38.884915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:33979 (local address 127.28.218.193:52543)
0806 22:50:38.885048 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:38.885060 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:38.885114 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:38.885230 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:38.885230 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:38.885404 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:38.885458 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:38.885460 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:38.886081 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:38.886081 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:38.886636 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:38.886638 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:38.886972 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:38.886973 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:38.887015 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:38.887519 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:50:38.887524 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:38.889506 (+ 1982us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.218.254:33979: Server oryx/127.28.218.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":84,"threads_started":1}
W20260806 22:50:38.889712 6874 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:50:38.885011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.193:52543 (local address 127.28.218.254:33979)
0806 22:50:38.885113 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:38.885115 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:38.885124 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:50:38.885158 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:38.885159 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:38.885168 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:38.885219 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:38.885523 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:38.886007 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:38.886718 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:38.886896 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:38.886941 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:38.889597 (+ 2656us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.218.193:52543: BlockingRecv error: recv got EOF from 127.28.218.193:52543 (error 108)
Metrics: {"server-negotiator.queue_time_us":81,"thread_start_us":50,"threads_started":1}
Aug 06 22:50:39 dist-test-slave-1xrh krb5kdc[5743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.218.194@KRBTEST.COM for oryx/127.28.218.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:50:39 dist-test-slave-1xrh krb5kdc[5743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.218.194@KRBTEST.COM for oryx/127.28.218.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:50:39.035614 6875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:39.030169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:33979 (local address 127.28.218.194:41107)
0806 22:50:39.030316 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:39.030327 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:39.030384 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:39.030596 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:39.030597 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:39.030820 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:39.030885 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:39.030887 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:39.031578 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:39.031578 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:39.032093 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:39.032094 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:39.032568 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:39.032570 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:39.032629 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:39.033140 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:50:39.033158 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:39.035531 (+ 2373us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.218.254:33979: Server oryx/127.28.218.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":65,"threads_started":1}
W20260806 22:50:39.035728 6874 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:50:39.030294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.194:41107 (local address 127.28.218.254:33979)
0806 22:50:39.030363 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:39.030366 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:39.030376 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:50:39.030480 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:39.030481 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:39.030496 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:39.030577 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:39.030956 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:39.031513 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:39.032215 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:39.032481 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:39.032554 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:39.035639 (+ 3085us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.218.194:41107: BlockingRecv error: recv got EOF from 127.28.218.194:41107 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 22:50:39.062304 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 6365
I20260806 22:50:39.068585 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 6530
I20260806 22:50:39.073921 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 6671
I20260806 22:50:39.078514 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 6291
2026-08-06T22:50:39Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17492 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-06T22:50:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:50:39Z Disabled control of system clock
I20260806 22:50:39.106884 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056586788339-29547-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056586788339-29547-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056586788339-29547-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.218.254:38371
--webserver_interface=127.28.218.254
--webserver_port=0
--builtin_ntp_servers=127.28.218.212:41171
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.218.254:38371
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260806 22:50:39.186039 6879 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:50:39.186257 6879 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:50:39.186290 6879 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:50:39.187501 6879 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:50:39.187568 6879 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:50:39.187589 6879 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:50:39.187613 6879 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:50:39.187654 6879 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:50:39.188952 6879 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:41171
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056586788339-29547-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056586788339-29547-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.218.254:38371
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.218.254:38371
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.218.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6879
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056586788339-29547-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:50:39.189229 6879 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:50:39.189493 6879 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:50:39.194285 6885 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:50:39.194357 6879 server_base.cc:1034] running on GCE node
W20260806 22:50:39.194320 6888 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:39.194284 6886 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:50:39.194789 6879 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:50:39.195075 6879 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:50:39.196280 6879 hybrid_clock.cc:648] HybridClock initialized: now 1786056639196266 us; error 56 us; skew 500 ppm
I20260806 22:50:39.197438 6879 webserver.cc:460] Webserver started at http://127.28.218.254:45283/ using document root <none> and password file <none>
I20260806 22:50:39.197664 6879 fs_manager.cc:359] Metadata directory not provided
I20260806 22:50:39.197731 6879 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:50:39.197882 6879 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:50:39.198920 6879 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056586788339-29547-0/minicluster-data/master-0/data/instance:
uuid: "f0ab20709f5f409abdee10fcf3e001ea"
format_stamp: "Formatted at 2026-08-06 22:50:39 on dist-test-slave-1xrh"
server_key: "18bbea62c8982c1575fc24a621210fc2"
server_key_iv: "7d3ca88ff5886aaa5663509ef6b471a6"
server_key_version: "encryptionkey@0"
I20260806 22:50:39.199306 6879 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056586788339-29547-0/minicluster-data/master-0/wal/instance:
uuid: "f0ab20709f5f409abdee10fcf3e001ea"
format_stamp: "Formatted at 2026-08-06 22:50:39 on dist-test-slave-1xrh"
server_key: "18bbea62c8982c1575fc24a621210fc2"
server_key_iv: "7d3ca88ff5886aaa5663509ef6b471a6"
server_key_version: "encryptionkey@0"
I20260806 22:50:39.200528 6879 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:50:39.201210 6895 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:39.201385 6879 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:39.201479 6879 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056586788339-29547-0/minicluster-data/master-0/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056586788339-29547-0/minicluster-data/master-0/wal
uuid: "f0ab20709f5f409abdee10fcf3e001ea"
format_stamp: "Formatted at 2026-08-06 22:50:39 on dist-test-slave-1xrh"
server_key: "18bbea62c8982c1575fc24a621210fc2"
server_key_iv: "7d3ca88ff5886aaa5663509ef6b471a6"
server_key_version: "encryptionkey@0"
I20260806 22:50:39.201555 6879 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056586788339-29547-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056586788339-29547-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056586788339-29547-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:50:39.228586 6879 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:50:39.228904 6879 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:50:39.229008 6879 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:50:39.233042 6879 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.254:38371
I20260806 22:50:39.233076 6950 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.254:38371 every 8 connection(s)
I20260806 22:50:39.233390 6879 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
I20260806 22:50:39.234063 6951 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:50:39.236542 6951 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f0ab20709f5f409abdee10fcf3e001ea: Bootstrap starting.
I20260806 22:50:39.237041 6951 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f0ab20709f5f409abdee10fcf3e001ea: Neither blocks nor log segments found. Creating new log.
I20260806 22:50:39.237270 6951 log.cc:824] T 00000000000000000000000000000000 P f0ab20709f5f409abdee10fcf3e001ea: Log is configured to *not* fsync() on all Append() calls
I20260806 22:50:39.237885 6951 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f0ab20709f5f409abdee10fcf3e001ea: No bootstrap required, opened a new log
I20260806 22:50:39.239446 6951 raft_consensus.cc:348] T 00000000000000000000000000000000 P f0ab20709f5f409abdee10fcf3e001ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0ab20709f5f409abdee10fcf3e001ea" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 38371 } }
I20260806 22:50:39.239571 6951 raft_consensus.cc:374] T 00000000000000000000000000000000 P f0ab20709f5f409abdee10fcf3e001ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:50:39.239593 6951 raft_consensus.cc:729] T 00000000000000000000000000000000 P f0ab20709f5f409abdee10fcf3e001ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0ab20709f5f409abdee10fcf3e001ea, State: Initialized, Role: FOLLOWER
I20260806 22:50:39.239715 6951 consensus_queue.cc:260] T 00000000000000000000000000000000 P f0ab20709f5f409abdee10fcf3e001ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0ab20709f5f409abdee10fcf3e001ea" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 38371 } }
I20260806 22:50:39.239790 6951 raft_consensus.cc:388] T 00000000000000000000000000000000 P f0ab20709f5f409abdee10fcf3e001ea [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:50:39.239811 6951 raft_consensus.cc:482] T 00000000000000000000000000000000 P f0ab20709f5f409abdee10fcf3e001ea [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:50:39.239838 6951 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f0ab20709f5f409abdee10fcf3e001ea [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:50:39.240288 6951 raft_consensus.cc:504] T 00000000000000000000000000000000 P f0ab20709f5f409abdee10fcf3e001ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0ab20709f5f409abdee10fcf3e001ea" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 38371 } }
I20260806 22:50:39.240365 6951 leader_election.cc:302] T 00000000000000000000000000000000 P f0ab20709f5f409abdee10fcf3e001ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f0ab20709f5f409abdee10fcf3e001ea; no voters:
I20260806 22:50:39.240479 6951 leader_election.cc:288] T 00000000000000000000000000000000 P f0ab20709f5f409abdee10fcf3e001ea [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:50:39.240556 6955 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f0ab20709f5f409abdee10fcf3e001ea [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:50:39.240727 6955 raft_consensus.cc:686] T 00000000000000000000000000000000 P f0ab20709f5f409abdee10fcf3e001ea [term 1 LEADER]: Becoming Leader. State: Replica: f0ab20709f5f409abdee10fcf3e001ea, State: Running, Role: LEADER
I20260806 22:50:39.240761 6951 sys_catalog.cc:564] T 00000000000000000000000000000000 P f0ab20709f5f409abdee10fcf3e001ea [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:50:39.240823 6955 consensus_queue.cc:237] T 00000000000000000000000000000000 P f0ab20709f5f409abdee10fcf3e001ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0ab20709f5f409abdee10fcf3e001ea" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 38371 } }
I20260806 22:50:39.241312 6956 sys_catalog.cc:455] T 00000000000000000000000000000000 P f0ab20709f5f409abdee10fcf3e001ea [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f0ab20709f5f409abdee10fcf3e001ea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0ab20709f5f409abdee10fcf3e001ea" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 38371 } } }
I20260806 22:50:39.241326 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 6879
I20260806 22:50:39.241381 6956 sys_catalog.cc:458] T 00000000000000000000000000000000 P f0ab20709f5f409abdee10fcf3e001ea [sys.catalog]: This master's current role is: LEADER
I20260806 22:50:39.241437 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056586788339-29547-0/minicluster-data/master-0/wal/instance
I20260806 22:50:39.241658 29547 external_mini_cluster.cc:1428] Setting key 3291c048e2b2063f5fd60e8c0b0b25e8
I20260806 22:50:39.241909 6958 sys_catalog.cc:455] T 00000000000000000000000000000000 P f0ab20709f5f409abdee10fcf3e001ea [sys.catalog]: SysCatalogTable state changed. Reason: New leader f0ab20709f5f409abdee10fcf3e001ea. Latest consensus state: current_term: 1 leader_uuid: "f0ab20709f5f409abdee10fcf3e001ea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0ab20709f5f409abdee10fcf3e001ea" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 38371 } } }
I20260806 22:50:39.241986 6958 sys_catalog.cc:458] T 00000000000000000000000000000000 P f0ab20709f5f409abdee10fcf3e001ea [sys.catalog]: This master's current role is: LEADER
I20260806 22:50:39.242180 6970 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:50:39.242668 6970 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:50:39.245077 6970 catalog_manager.cc:1347] Generated new cluster ID: 7e9abd1177964a5c8bc9b7510049ec55
I20260806 22:50:39.245131 6970 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:50:39.247502 6975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:39.242337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42174 (local address 127.28.218.254:38371)
0806 22:50:39.242722 (+ 385us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:39.242731 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:39.242763 (+ 32us) server_negotiation.cc:407] Connection header received
0806 22:50:39.242866 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:39.242871 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:39.242931 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:39.243068 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:39.243309 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:39.244639 (+ 1330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:39.245567 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:39.246010 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:39.246143 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:39.246815 (+ 672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:50:39.246818 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:50:39.246819 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:50:39.246999 (+ 180us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:50:39.247030 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:50:39.247040 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:50:39.247113 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:39.247223 (+ 110us) server_negotiation.cc:299] Negotiation successful
0806 22:50:39.247316 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":319,"thread_start_us":69,"threads_started":1}
I20260806 22:50:39.266896 6970 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:50:39.267606 6970 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:50:39.274574 6970 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f0ab20709f5f409abdee10fcf3e001ea: Generated new TSK 0
I20260806 22:50:39.274777 6970 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:50:39.284310 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056586788339-29547-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056586788339-29547-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056586788339-29547-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056586788339-29547-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.218.193:0
--local_ip_for_outbound_sockets=127.28.218.193
--webserver_interface=127.28.218.193
--webserver_port=0
--tserver_master_addrs=127.28.218.254:38371
--builtin_ntp_servers=127.28.218.212:41171
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260806 22:50:39.368468 6978 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:50:39.368695 6978 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:50:39.368731 6978 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:50:39.369829 6978 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:50:39.369891 6978 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:50:39.369930 6978 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.218.193
I20260806 22:50:39.371316 6978 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:41171
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056586788339-29547-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056586788339-29547-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.218.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056586788339-29547-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.218.193
--webserver_port=0
--tserver_master_addrs=127.28.218.254:38371
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6978
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.218.193
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056586788339-29547-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:50:39.371582 6978 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:50:39.371878 6978 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:50:39.376844 6985 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:39.376963 6987 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:39.376844 6984 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:50:39.377101 6978 server_base.cc:1034] running on GCE node
I20260806 22:50:39.377336 6978 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:50:39.377588 6978 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:50:39.378747 6978 hybrid_clock.cc:648] HybridClock initialized: now 1786056639378741 us; error 36 us; skew 500 ppm
I20260806 22:50:39.380103 6978 webserver.cc:460] Webserver started at http://127.28.218.193:37015/ using document root <none> and password file <none>
I20260806 22:50:39.380352 6978 fs_manager.cc:359] Metadata directory not provided
I20260806 22:50:39.380438 6978 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:50:39.380591 6978 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:50:39.381641 6978 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056586788339-29547-0/minicluster-data/ts-0/data/instance:
uuid: "acad0ee9943f4d608d559a60973102b0"
format_stamp: "Formatted at 2026-08-06 22:50:39 on dist-test-slave-1xrh"
server_key: "b10d08af8a412e6524d44bad797dbb84"
server_key_iv: "7226948c8bf0a56469b6ccfa987311e9"
server_key_version: "encryptionkey@0"
I20260806 22:50:39.382009 6978 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056586788339-29547-0/minicluster-data/ts-0/wal/instance:
uuid: "acad0ee9943f4d608d559a60973102b0"
format_stamp: "Formatted at 2026-08-06 22:50:39 on dist-test-slave-1xrh"
server_key: "b10d08af8a412e6524d44bad797dbb84"
server_key_iv: "7226948c8bf0a56469b6ccfa987311e9"
server_key_version: "encryptionkey@0"
I20260806 22:50:39.383301 6978 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:50:39.384018 6993 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:39.384182 6978 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:50:39.384279 6978 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056586788339-29547-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056586788339-29547-0/minicluster-data/ts-0/wal
uuid: "acad0ee9943f4d608d559a60973102b0"
format_stamp: "Formatted at 2026-08-06 22:50:39 on dist-test-slave-1xrh"
server_key: "b10d08af8a412e6524d44bad797dbb84"
server_key_iv: "7226948c8bf0a56469b6ccfa987311e9"
server_key_version: "encryptionkey@0"
I20260806 22:50:39.384382 6978 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056586788339-29547-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056586788339-29547-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056586788339-29547-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:50:39.416494 6978 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:50:39.416797 6978 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:50:39.416957 6978 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:50:39.417174 6978 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:50:39.417487 6978 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:50:39.417541 6978 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:39.417575 6978 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:50:39.417603 6978 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:39.423331 6978 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.193:39691
I20260806 22:50:39.423403 7113 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.193:39691 every 8 connection(s)
I20260806 22:50:39.423700 6978 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056586788339-29547-0/minicluster-data/ts-0/data/info.pb
I20260806 22:50:39.429008 6975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:39.424366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.193:50693 (local address 127.28.218.254:38371)
0806 22:50:39.424431 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:39.424433 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:39.424669 (+ 236us) server_negotiation.cc:407] Connection header received
0806 22:50:39.425209 (+ 540us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:39.425210 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:39.425225 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:39.425303 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:39.425967 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:39.426474 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:39.427523 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:39.427753 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:39.427834 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:39.428690 (+ 856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:50:39.428691 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:50:39.428691 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:50:39.428696 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:50:39.428711 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:50:39.428716 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:50:39.428759 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:39.428893 (+ 134us) server_negotiation.cc:299] Negotiation successful
0806 22:50:39.428929 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 22:50:39.429217 7116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:39.424367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:38371 (local address 127.28.218.193:50693)
0806 22:50:39.424556 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:39.424588 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:39.425094 (+ 506us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:39.425384 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:39.425387 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:39.425397 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:39.425865 (+ 468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:39.425869 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:39.426569 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:39.426570 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:39.427430 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:39.427433 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:39.427845 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:39.427846 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:39.427974 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:39.428533 (+ 559us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:50:39.428534 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:39.428560 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:50:39.428581 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:50:39.428771 (+ 190us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:50:39.428775 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:50:39.428778 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:50:39.428862 (+ 84us) client_negotiation.cc:240] Negotiation successful
0806 22:50:39.428963 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":92,"threads_started":1}
I20260806 22:50:39.429666 7114 heartbeater.cc:347] Connected to a master server at 127.28.218.254:38371
I20260806 22:50:39.429770 7114 heartbeater.cc:464] Registering TS with master...
I20260806 22:50:39.429958 7114 heartbeater.cc:511] Master 127.28.218.254:38371 requested a full tablet report, sending...
I20260806 22:50:39.429987 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 6978
I20260806 22:50:39.430079 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056586788339-29547-0/minicluster-data/ts-0/wal/instance
I20260806 22:50:39.430306 29547 external_mini_cluster.cc:1428] Setting key 9b272285a06b044f0efe6187535791ae
I20260806 22:50:39.430526 6913 ts_manager.cc:194] Registered new tserver with Master: acad0ee9943f4d608d559a60973102b0 (127.28.218.193:39691)
I20260806 22:50:39.431413 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056586788339-29547-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056586788339-29547-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056586788339-29547-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056586788339-29547-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.218.194:0
--local_ip_for_outbound_sockets=127.28.218.194
--webserver_interface=127.28.218.194
--webserver_port=0
--tserver_master_addrs=127.28.218.254:38371
--builtin_ntp_servers=127.28.218.212:41171
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260806 22:50:39.431433 6913 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.28.218.193:50693
W20260806 22:50:39.510573 7117 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:50:39.510747 7117 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:50:39.510766 7117 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:50:39.511878 7117 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:50:39.511914 7117 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:50:39.511937 7117 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.218.194
I20260806 22:50:39.513108 7117 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:41171
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056586788339-29547-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056586788339-29547-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.218.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056586788339-29547-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.218.194
--webserver_port=0
--tserver_master_addrs=127.28.218.254:38371
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7117
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.218.194
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056586788339-29547-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:50:39.513268 7117 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:50:39.513501 7117 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:50:39.518261 7122 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:39.518261 7124 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:39.518317 7126 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:50:39.518415 7117 server_base.cc:1034] running on GCE node
I20260806 22:50:39.518640 7117 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:50:39.518872 7117 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:50:39.520049 7117 hybrid_clock.cc:648] HybridClock initialized: now 1786056639520021 us; error 53 us; skew 500 ppm
I20260806 22:50:39.521189 7117 webserver.cc:460] Webserver started at http://127.28.218.194:36871/ using document root <none> and password file <none>
I20260806 22:50:39.521427 7117 fs_manager.cc:359] Metadata directory not provided
I20260806 22:50:39.521492 7117 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:50:39.521623 7117 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:50:39.522627 7117 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056586788339-29547-0/minicluster-data/ts-1/data/instance:
uuid: "1d564c00e37b4ee7aa8e25f8b70305d7"
format_stamp: "Formatted at 2026-08-06 22:50:39 on dist-test-slave-1xrh"
server_key: "da2877f0ad1861b4b173ceb409714cad"
server_key_iv: "34082acbf9963673b599b4416990b73f"
server_key_version: "encryptionkey@0"
I20260806 22:50:39.522969 7117 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056586788339-29547-0/minicluster-data/ts-1/wal/instance:
uuid: "1d564c00e37b4ee7aa8e25f8b70305d7"
format_stamp: "Formatted at 2026-08-06 22:50:39 on dist-test-slave-1xrh"
server_key: "da2877f0ad1861b4b173ceb409714cad"
server_key_iv: "34082acbf9963673b599b4416990b73f"
server_key_version: "encryptionkey@0"
I20260806 22:50:39.524248 7117 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:50:39.524919 7133 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:39.525098 7117 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:50:39.525193 7117 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056586788339-29547-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056586788339-29547-0/minicluster-data/ts-1/wal
uuid: "1d564c00e37b4ee7aa8e25f8b70305d7"
format_stamp: "Formatted at 2026-08-06 22:50:39 on dist-test-slave-1xrh"
server_key: "da2877f0ad1861b4b173ceb409714cad"
server_key_iv: "34082acbf9963673b599b4416990b73f"
server_key_version: "encryptionkey@0"
I20260806 22:50:39.525278 7117 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056586788339-29547-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056586788339-29547-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056586788339-29547-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:50:39.537367 7117 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:50:39.537631 7117 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:50:39.537777 7117 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:50:39.537983 7117 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:50:39.538295 7117 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:50:39.538348 7117 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:39.538388 7117 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:50:39.538414 7117 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:39.544236 7117 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.194:35313
I20260806 22:50:39.544289 7248 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.194:35313 every 8 connection(s)
I20260806 22:50:39.544576 7117 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056586788339-29547-0/minicluster-data/ts-1/data/info.pb
I20260806 22:50:39.546429 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 7117
I20260806 22:50:39.546502 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056586788339-29547-0/minicluster-data/ts-1/wal/instance
I20260806 22:50:39.546713 29547 external_mini_cluster.cc:1428] Setting key f0025dda87324b9e9b59e49e235b6687
I20260806 22:50:39.547713 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056586788339-29547-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056586788339-29547-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056586788339-29547-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056586788339-29547-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.218.195:0
--local_ip_for_outbound_sockets=127.28.218.195
--webserver_interface=127.28.218.195
--webserver_port=0
--tserver_master_addrs=127.28.218.254:38371
--builtin_ntp_servers=127.28.218.212:41171
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260806 22:50:39.549907 6975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:39.545235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.194:44215 (local address 127.28.218.254:38371)
0806 22:50:39.545312 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:39.545314 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:39.545501 (+ 187us) server_negotiation.cc:407] Connection header received
0806 22:50:39.546003 (+ 502us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:39.546004 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:39.546020 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:39.546073 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:39.546756 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:39.547456 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:39.548510 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:39.548716 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:39.548789 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:39.549651 (+ 862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:50:39.549652 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:50:39.549653 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:50:39.549657 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:50:39.549671 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:50:39.549675 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:50:39.549721 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:39.549828 (+ 107us) server_negotiation.cc:299] Negotiation successful
0806 22:50:39.549852 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 22:50:39.550149 7251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:39.545196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:38371 (local address 127.28.218.194:44215)
0806 22:50:39.545384 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:39.545415 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:39.545887 (+ 472us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:39.546135 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:39.546138 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:39.546145 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:39.546662 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:39.546667 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:39.547531 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:39.547532 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:39.548408 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:39.548411 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:39.548794 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:39.548796 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:39.548928 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:39.549488 (+ 560us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:50:39.549488 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:39.549527 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:50:39.549556 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:50:39.549734 (+ 178us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:50:39.549738 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:50:39.549741 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:50:39.549819 (+ 78us) client_negotiation.cc:240] Negotiation successful
0806 22:50:39.549928 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":97,"threads_started":1}
I20260806 22:50:39.550583 7249 heartbeater.cc:347] Connected to a master server at 127.28.218.254:38371
I20260806 22:50:39.550671 7249 heartbeater.cc:464] Registering TS with master...
I20260806 22:50:39.550844 7249 heartbeater.cc:511] Master 127.28.218.254:38371 requested a full tablet report, sending...
I20260806 22:50:39.551198 6913 ts_manager.cc:194] Registered new tserver with Master: 1d564c00e37b4ee7aa8e25f8b70305d7 (127.28.218.194:35313)
I20260806 22:50:39.551590 6913 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.28.218.194:44215
W20260806 22:50:39.628387 7252 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:50:39.628603 7252 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:50:39.628635 7252 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:50:39.629791 7252 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:50:39.629853 7252 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:50:39.629891 7252 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.218.195
I20260806 22:50:39.631232 7252 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:41171
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056586788339-29547-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056586788339-29547-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.218.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056586788339-29547-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.218.195
--webserver_port=0
--tserver_master_addrs=127.28.218.254:38371
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7252
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.218.195
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056586788339-29547-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:50:39.631465 7252 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:50:39.631734 7252 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:50:39.636682 7257 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:39.636713 7258 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:50:39.636912 7252 server_base.cc:1034] running on GCE node
W20260806 22:50:39.636934 7260 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:50:39.637138 7252 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:50:39.637368 7252 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:50:39.638523 7252 hybrid_clock.cc:648] HybridClock initialized: now 1786056639638515 us; error 32 us; skew 500 ppm
I20260806 22:50:39.639791 7252 webserver.cc:460] Webserver started at http://127.28.218.195:42549/ using document root <none> and password file <none>
I20260806 22:50:39.640015 7252 fs_manager.cc:359] Metadata directory not provided
I20260806 22:50:39.640110 7252 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:50:39.640250 7252 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:50:39.641340 7252 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056586788339-29547-0/minicluster-data/ts-2/data/instance:
uuid: "5ab4d2c6a7ee4b2fb2700a0b1848bfea"
format_stamp: "Formatted at 2026-08-06 22:50:39 on dist-test-slave-1xrh"
server_key: "718299cd20135c5016c9ad7a98bb191f"
server_key_iv: "3cd6822a71ae9867c33a389c086862b0"
server_key_version: "encryptionkey@0"
I20260806 22:50:39.641737 7252 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056586788339-29547-0/minicluster-data/ts-2/wal/instance:
uuid: "5ab4d2c6a7ee4b2fb2700a0b1848bfea"
format_stamp: "Formatted at 2026-08-06 22:50:39 on dist-test-slave-1xrh"
server_key: "718299cd20135c5016c9ad7a98bb191f"
server_key_iv: "3cd6822a71ae9867c33a389c086862b0"
server_key_version: "encryptionkey@0"
I20260806 22:50:39.643153 7252 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:50:39.643945 7267 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:39.644127 7252 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:50:39.644222 7252 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056586788339-29547-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056586788339-29547-0/minicluster-data/ts-2/wal
uuid: "5ab4d2c6a7ee4b2fb2700a0b1848bfea"
format_stamp: "Formatted at 2026-08-06 22:50:39 on dist-test-slave-1xrh"
server_key: "718299cd20135c5016c9ad7a98bb191f"
server_key_iv: "3cd6822a71ae9867c33a389c086862b0"
server_key_version: "encryptionkey@0"
I20260806 22:50:39.644307 7252 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056586788339-29547-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056586788339-29547-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056586788339-29547-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:50:39.663605 7252 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:50:39.663899 7252 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:50:39.664045 7252 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:50:39.664259 7252 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:50:39.664582 7252 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:50:39.664635 7252 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:39.664669 7252 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:50:39.664695 7252 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:39.670274 7252 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.195:46653
I20260806 22:50:39.670313 7384 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.195:46653 every 8 connection(s)
I20260806 22:50:39.670635 7252 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056586788339-29547-0/minicluster-data/ts-2/data/info.pb
I20260806 22:50:39.673533 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 7252
I20260806 22:50:39.673646 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056586788339-29547-0/minicluster-data/ts-2/wal/instance
I20260806 22:50:39.673823 29547 external_mini_cluster.cc:1428] Setting key 5ba8b3e70a39767a3ce38750b2913335
I20260806 22:50:39.675796 6975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:39.671320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.195:35027 (local address 127.28.218.254:38371)
0806 22:50:39.671407 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:39.671409 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:39.671623 (+ 214us) server_negotiation.cc:407] Connection header received
0806 22:50:39.672070 (+ 447us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:39.672071 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:39.672081 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:39.672133 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:39.672716 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:39.673209 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:39.674364 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:39.674675 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:39.674744 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:39.675575 (+ 831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:50:39.675576 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:50:39.675576 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:50:39.675580 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:50:39.675593 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:50:39.675597 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:50:39.675663 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:39.675715 (+ 52us) server_negotiation.cc:299] Negotiation successful
0806 22:50:39.675741 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 22:50:39.676005 7387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:39.671302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:38371 (local address 127.28.218.195:35027)
0806 22:50:39.671511 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:39.671543 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:39.671989 (+ 446us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:39.672182 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:39.672185 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:39.672205 (+ 20us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:39.672653 (+ 448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:39.672656 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:39.673296 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:39.673297 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:39.674285 (+ 988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:39.674287 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:39.674763 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:39.674764 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:39.674917 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:39.675436 (+ 519us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:50:39.675437 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:39.675463 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:50:39.675484 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:50:39.675666 (+ 182us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:50:39.675668 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:50:39.675670 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 22:50:39.675710 (+ 40us) client_negotiation.cc:240] Negotiation successful
0806 22:50:39.675779 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":108,"threads_started":1}
I20260806 22:50:39.676474 7385 heartbeater.cc:347] Connected to a master server at 127.28.218.254:38371
I20260806 22:50:39.676559 7385 heartbeater.cc:464] Registering TS with master...
I20260806 22:50:39.676692 7385 heartbeater.cc:511] Master 127.28.218.254:38371 requested a full tablet report, sending...
I20260806 22:50:39.677000 6913 ts_manager.cc:194] Registered new tserver with Master: 5ab4d2c6a7ee4b2fb2700a0b1848bfea (127.28.218.195:46653)
I20260806 22:50:39.677356 6913 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.28.218.195:35027
I20260806 22:50:39.685603 29547 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 22:50:39.687597 6975 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:50:39.686940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45808 (local address 127.28.218.254:38371)
0806 22:50:39.686989 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:39.686991 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:39.687240 (+ 249us) server_negotiation.cc:407] Connection header received
0806 22:50:39.687277 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:39.687278 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:39.687288 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:39.687367 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:39.687536 (+ 169us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45808: BlockingRecv error: recv got EOF from 127.0.0.1:45808 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
I20260806 22:50:39.687686 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 6978
I20260806 22:50:39.694938 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 7117
I20260806 22:50:39.699836 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 7252
I20260806 22:50:39.705397 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 6879
2026-08-06T22:50:39Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (626 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-06T22:50:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:50:39Z Disabled control of system clock
I20260806 22:50:39.733341 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056586788339-29547-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056586788339-29547-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056586788339-29547-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.218.254:38049
--webserver_interface=127.28.218.254
--webserver_port=0
--builtin_ntp_servers=127.28.218.212:40399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.218.254:38049
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260806 22:50:39.812971 7399 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:50:39.813158 7399 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:50:39.813179 7399 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:50:39.814316 7399 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:50:39.814353 7399 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:50:39.814365 7399 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:50:39.814401 7399 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:50:39.814427 7399 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:50:39.815797 7399 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:40399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056586788339-29547-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056586788339-29547-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.218.254:38049
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.218.254:38049
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.218.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7399
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056586788339-29547-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:50:39.816037 7399 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:50:39.816314 7399 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:50:39.820973 7404 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:39.821002 7405 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:50:39.821123 7399 server_base.cc:1034] running on GCE node
W20260806 22:50:39.821235 7407 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:50:39.821483 7399 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:50:39.821732 7399 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:50:39.822902 7399 hybrid_clock.cc:648] HybridClock initialized: now 1786056639822890 us; error 29 us; skew 500 ppm
I20260806 22:50:39.824136 7399 webserver.cc:460] Webserver started at http://127.28.218.254:42639/ using document root <none> and password file <none>
I20260806 22:50:39.824365 7399 fs_manager.cc:359] Metadata directory not provided
I20260806 22:50:39.824409 7399 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:50:39.824616 7399 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:50:39.825588 7399 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056586788339-29547-0/minicluster-data/master-0/data/instance:
uuid: "0373792d77b14e608eb21c04f1c2a244"
format_stamp: "Formatted at 2026-08-06 22:50:39 on dist-test-slave-1xrh"
server_key: "9eaa35c849155ebf7caadfc24f787db1"
server_key_iv: "71e40e1663ecf9dda38a2cbd23e74cc2"
server_key_version: "encryptionkey@0"
I20260806 22:50:39.825938 7399 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056586788339-29547-0/minicluster-data/master-0/wal/instance:
uuid: "0373792d77b14e608eb21c04f1c2a244"
format_stamp: "Formatted at 2026-08-06 22:50:39 on dist-test-slave-1xrh"
server_key: "9eaa35c849155ebf7caadfc24f787db1"
server_key_iv: "71e40e1663ecf9dda38a2cbd23e74cc2"
server_key_version: "encryptionkey@0"
I20260806 22:50:39.827214 7399 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:50:39.827889 7414 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:39.828084 7399 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:50:39.828174 7399 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056586788339-29547-0/minicluster-data/master-0/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056586788339-29547-0/minicluster-data/master-0/wal
uuid: "0373792d77b14e608eb21c04f1c2a244"
format_stamp: "Formatted at 2026-08-06 22:50:39 on dist-test-slave-1xrh"
server_key: "9eaa35c849155ebf7caadfc24f787db1"
server_key_iv: "71e40e1663ecf9dda38a2cbd23e74cc2"
server_key_version: "encryptionkey@0"
I20260806 22:50:39.828243 7399 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056586788339-29547-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056586788339-29547-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056586788339-29547-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:50:39.831622 7399 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:50:39.831871 7399 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:50:39.832031 7399 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:50:39.835909 7399 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.254:38049
I20260806 22:50:39.835979 7468 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.254:38049 every 8 connection(s)
I20260806 22:50:39.836261 7399 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
I20260806 22:50:39.836930 7469 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:50:39.837353 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 7399
I20260806 22:50:39.837435 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056586788339-29547-0/minicluster-data/master-0/wal/instance
I20260806 22:50:39.837639 29547 external_mini_cluster.cc:1428] Setting key b4801fe2633f74955680f5e86552579b
I20260806 22:50:39.840343 7472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:39.838193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45260 (local address 127.28.218.254:38049)
0806 22:50:39.838415 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:39.838423 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:39.838456 (+ 33us) server_negotiation.cc:407] Connection header received
0806 22:50:39.839069 (+ 613us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:39.839078 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:39.839133 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:39.839296 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:39.839926 (+ 630us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:50:39.839929 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:50:39.839929 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:50:39.839934 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:50:39.839978 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:50:39.839991 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:50:39.840072 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:39.840102 (+ 30us) server_negotiation.cc:299] Negotiation successful
0806 22:50:39.840155 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":75,"threads_started":1}
I20260806 22:50:39.840946 7469 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0373792d77b14e608eb21c04f1c2a244: Bootstrap starting.
I20260806 22:50:39.841524 7469 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0373792d77b14e608eb21c04f1c2a244: Neither blocks nor log segments found. Creating new log.
I20260806 22:50:39.841749 7469 log.cc:824] T 00000000000000000000000000000000 P 0373792d77b14e608eb21c04f1c2a244: Log is configured to *not* fsync() on all Append() calls
I20260806 22:50:39.842391 7469 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0373792d77b14e608eb21c04f1c2a244: No bootstrap required, opened a new log
I20260806 22:50:39.843832 7469 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0373792d77b14e608eb21c04f1c2a244 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0373792d77b14e608eb21c04f1c2a244" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 38049 } }
I20260806 22:50:39.843984 7469 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0373792d77b14e608eb21c04f1c2a244 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:50:39.844007 7469 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0373792d77b14e608eb21c04f1c2a244 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0373792d77b14e608eb21c04f1c2a244, State: Initialized, Role: FOLLOWER
I20260806 22:50:39.844085 7469 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0373792d77b14e608eb21c04f1c2a244 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0373792d77b14e608eb21c04f1c2a244" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 38049 } }
I20260806 22:50:39.844132 7469 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0373792d77b14e608eb21c04f1c2a244 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:50:39.844161 7469 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0373792d77b14e608eb21c04f1c2a244 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:50:39.844192 7469 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0373792d77b14e608eb21c04f1c2a244 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:50:39.844647 7469 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0373792d77b14e608eb21c04f1c2a244 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0373792d77b14e608eb21c04f1c2a244" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 38049 } }
I20260806 22:50:39.844727 7469 leader_election.cc:302] T 00000000000000000000000000000000 P 0373792d77b14e608eb21c04f1c2a244 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0373792d77b14e608eb21c04f1c2a244; no voters:
I20260806 22:50:39.844848 7469 leader_election.cc:288] T 00000000000000000000000000000000 P 0373792d77b14e608eb21c04f1c2a244 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:50:39.845095 7469 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0373792d77b14e608eb21c04f1c2a244 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:50:39.845151 7474 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0373792d77b14e608eb21c04f1c2a244 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:50:39.845271 7474 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0373792d77b14e608eb21c04f1c2a244 [term 1 LEADER]: Becoming Leader. State: Replica: 0373792d77b14e608eb21c04f1c2a244, State: Running, Role: LEADER
I20260806 22:50:39.845368 7474 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0373792d77b14e608eb21c04f1c2a244 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0373792d77b14e608eb21c04f1c2a244" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 38049 } }
I20260806 22:50:39.845724 7475 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0373792d77b14e608eb21c04f1c2a244 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0373792d77b14e608eb21c04f1c2a244" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0373792d77b14e608eb21c04f1c2a244" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 38049 } } }
I20260806 22:50:39.845844 7475 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0373792d77b14e608eb21c04f1c2a244 [sys.catalog]: This master's current role is: LEADER
I20260806 22:50:39.845726 7476 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0373792d77b14e608eb21c04f1c2a244 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0373792d77b14e608eb21c04f1c2a244. Latest consensus state: current_term: 1 leader_uuid: "0373792d77b14e608eb21c04f1c2a244" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0373792d77b14e608eb21c04f1c2a244" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 38049 } } }
I20260806 22:50:39.845937 7476 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0373792d77b14e608eb21c04f1c2a244 [sys.catalog]: This master's current role is: LEADER
I20260806 22:50:39.846284 7485 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:50:39.846742 7485 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:50:39.848193 7485 catalog_manager.cc:1347] Generated new cluster ID: 4356a214f7c647c195cec5397c54fbfc
I20260806 22:50:39.848263 7485 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:50:39.859195 7485 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:50:39.859344 7485 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:50:39.862267 7485 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0373792d77b14e608eb21c04f1c2a244: Generated new TSK 0
I20260806 22:50:39.862437 7485 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:50:39.866458 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056586788339-29547-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056586788339-29547-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056586788339-29547-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056586788339-29547-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.218.193:0
--local_ip_for_outbound_sockets=127.28.218.193
--webserver_interface=127.28.218.193
--webserver_port=0
--tserver_master_addrs=127.28.218.254:38049
--builtin_ntp_servers=127.28.218.212:40399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260806 22:50:39.946967 7495 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:50:39.947294 7495 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:50:39.947336 7495 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:50:39.948793 7495 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:50:39.948856 7495 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:50:39.948899 7495 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.218.193
I20260806 22:50:39.950202 7495 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:40399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056586788339-29547-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056586788339-29547-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.218.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056586788339-29547-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.218.193
--webserver_port=0
--tserver_master_addrs=127.28.218.254:38049
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7495
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.218.193
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056586788339-29547-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:50:39.950456 7495 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:50:39.950747 7495 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:50:39.956585 7501 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:39.956606 7503 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:50:39.956755 7495 server_base.cc:1034] running on GCE node
W20260806 22:50:39.956600 7500 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:50:39.957016 7495 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:50:39.957258 7495 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:50:39.958422 7495 hybrid_clock.cc:648] HybridClock initialized: now 1786056639958397 us; error 40 us; skew 500 ppm
I20260806 22:50:39.959789 7495 webserver.cc:460] Webserver started at http://127.28.218.193:39759/ using document root <none> and password file <none>
I20260806 22:50:39.960062 7495 fs_manager.cc:359] Metadata directory not provided
I20260806 22:50:39.960139 7495 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:50:39.960296 7495 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:50:39.961351 7495 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056586788339-29547-0/minicluster-data/ts-0/data/instance:
uuid: "324b2e861c2b4dee8cde74d5554038b7"
format_stamp: "Formatted at 2026-08-06 22:50:39 on dist-test-slave-1xrh"
server_key: "af6acd6b56476cd193c5d808aeada1b3"
server_key_iv: "38b16e23ed02f371d2c2831f21581e09"
server_key_version: "encryptionkey@0"
I20260806 22:50:39.961771 7495 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056586788339-29547-0/minicluster-data/ts-0/wal/instance:
uuid: "324b2e861c2b4dee8cde74d5554038b7"
format_stamp: "Formatted at 2026-08-06 22:50:39 on dist-test-slave-1xrh"
server_key: "af6acd6b56476cd193c5d808aeada1b3"
server_key_iv: "38b16e23ed02f371d2c2831f21581e09"
server_key_version: "encryptionkey@0"
I20260806 22:50:39.963114 7495 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:50:39.963977 7509 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:39.964180 7495 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:50:39.964279 7495 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056586788339-29547-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056586788339-29547-0/minicluster-data/ts-0/wal
uuid: "324b2e861c2b4dee8cde74d5554038b7"
format_stamp: "Formatted at 2026-08-06 22:50:39 on dist-test-slave-1xrh"
server_key: "af6acd6b56476cd193c5d808aeada1b3"
server_key_iv: "38b16e23ed02f371d2c2831f21581e09"
server_key_version: "encryptionkey@0"
I20260806 22:50:39.964368 7495 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056586788339-29547-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056586788339-29547-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056586788339-29547-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:50:39.967710 7495 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:50:39.967999 7495 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:50:39.968154 7495 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:50:39.968394 7495 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:50:39.968753 7495 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:50:39.968806 7495 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:39.968847 7495 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:50:39.968873 7495 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:39.975167 7495 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.193:37351
I20260806 22:50:39.975224 7625 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.193:37351 every 8 connection(s)
I20260806 22:50:39.975544 7495 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056586788339-29547-0/minicluster-data/ts-0/data/info.pb
I20260806 22:50:39.978546 7472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:39.976260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.193:54889 (local address 127.28.218.254:38049)
0806 22:50:39.976308 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:39.976310 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:39.976514 (+ 204us) server_negotiation.cc:407] Connection header received
0806 22:50:39.977014 (+ 500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:39.977016 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:39.977022 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:39.977089 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:39.978350 (+ 1261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:50:39.978351 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:50:39.978352 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:50:39.978356 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:50:39.978372 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:50:39.978376 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:50:39.978427 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:39.978482 (+ 55us) server_negotiation.cc:299] Negotiation successful
0806 22:50:39.978494 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 22:50:39.978744 7628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:39.976246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:38049 (local address 127.28.218.193:54889)
0806 22:50:39.976402 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:39.976436 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:39.976888 (+ 452us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:39.977141 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:39.977144 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:39.977152 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:39.978157 (+ 1005us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:50:39.978157 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:39.978186 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:50:39.978217 (+ 31us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:50:39.978432 (+ 215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:50:39.978435 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:50:39.978437 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 22:50:39.978488 (+ 51us) client_negotiation.cc:240] Negotiation successful
0806 22:50:39.978542 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":76,"threads_started":1}
I20260806 22:50:39.979486 7626 heartbeater.cc:347] Connected to a master server at 127.28.218.254:38049
I20260806 22:50:39.979581 7626 heartbeater.cc:464] Registering TS with master...
I20260806 22:50:39.979748 7626 heartbeater.cc:511] Master 127.28.218.254:38049 requested a full tablet report, sending...
I20260806 22:50:39.980257 7432 ts_manager.cc:194] Registered new tserver with Master: 324b2e861c2b4dee8cde74d5554038b7 (127.28.218.193:37351)
I20260806 22:50:39.981841 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 7495
I20260806 22:50:39.981982 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056586788339-29547-0/minicluster-data/ts-0/wal/instance
I20260806 22:50:39.982196 29547 external_mini_cluster.cc:1428] Setting key 8540e7417c6d46fbb9eff22284878b99
I20260806 22:50:39.983070 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056586788339-29547-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056586788339-29547-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056586788339-29547-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056586788339-29547-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.218.194:0
--local_ip_for_outbound_sockets=127.28.218.194
--webserver_interface=127.28.218.194
--webserver_port=0
--tserver_master_addrs=127.28.218.254:38049
--builtin_ntp_servers=127.28.218.212:40399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260806 22:50:40.065696 7629 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:50:40.065928 7629 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:50:40.065966 7629 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:50:40.067214 7629 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:50:40.067270 7629 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:50:40.067313 7629 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.218.194
I20260806 22:50:40.068780 7629 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:40399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056586788339-29547-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056586788339-29547-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.218.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056586788339-29547-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.218.194
--webserver_port=0
--tserver_master_addrs=127.28.218.254:38049
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7629
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.218.194
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056586788339-29547-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:50:40.069044 7629 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:50:40.069329 7629 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:50:40.074206 7635 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:40.074220 7634 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:40.074221 7637 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:50:40.074533 7629 server_base.cc:1034] running on GCE node
I20260806 22:50:40.074756 7629 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:50:40.074988 7629 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:50:40.076150 7629 hybrid_clock.cc:648] HybridClock initialized: now 1786056640076136 us; error 25 us; skew 500 ppm
I20260806 22:50:40.077339 7629 webserver.cc:460] Webserver started at http://127.28.218.194:45511/ using document root <none> and password file <none>
I20260806 22:50:40.077610 7629 fs_manager.cc:359] Metadata directory not provided
I20260806 22:50:40.077680 7629 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:50:40.077819 7629 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:50:40.078884 7629 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056586788339-29547-0/minicluster-data/ts-1/data/instance:
uuid: "b93627e1a7f446a1b54cd27824243576"
format_stamp: "Formatted at 2026-08-06 22:50:40 on dist-test-slave-1xrh"
server_key: "8fa2a82f18c55e664217d49b82aa435e"
server_key_iv: "a668b91f987405342635255be8255b98"
server_key_version: "encryptionkey@0"
I20260806 22:50:40.079277 7629 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056586788339-29547-0/minicluster-data/ts-1/wal/instance:
uuid: "b93627e1a7f446a1b54cd27824243576"
format_stamp: "Formatted at 2026-08-06 22:50:40 on dist-test-slave-1xrh"
server_key: "8fa2a82f18c55e664217d49b82aa435e"
server_key_iv: "a668b91f987405342635255be8255b98"
server_key_version: "encryptionkey@0"
I20260806 22:50:40.080515 7629 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:50:40.081199 7643 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:40.081358 7629 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:50:40.081445 7629 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056586788339-29547-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056586788339-29547-0/minicluster-data/ts-1/wal
uuid: "b93627e1a7f446a1b54cd27824243576"
format_stamp: "Formatted at 2026-08-06 22:50:40 on dist-test-slave-1xrh"
server_key: "8fa2a82f18c55e664217d49b82aa435e"
server_key_iv: "a668b91f987405342635255be8255b98"
server_key_version: "encryptionkey@0"
I20260806 22:50:40.081539 7629 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056586788339-29547-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056586788339-29547-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056586788339-29547-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:50:40.084903 7629 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:50:40.085155 7629 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:50:40.085300 7629 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:50:40.085515 7629 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:50:40.085812 7629 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:50:40.085865 7629 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:40.085904 7629 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:50:40.085932 7629 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:40.091594 7629 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.194:45263
I20260806 22:50:40.091641 7763 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.194:45263 every 8 connection(s)
I20260806 22:50:40.091933 7629 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056586788339-29547-0/minicluster-data/ts-1/data/info.pb
I20260806 22:50:40.094691 7472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:40.092548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.194:37641 (local address 127.28.218.254:38049)
0806 22:50:40.092611 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:40.092613 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:40.092844 (+ 231us) server_negotiation.cc:407] Connection header received
0806 22:50:40.093301 (+ 457us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:40.093302 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:40.093309 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:40.093364 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:40.094429 (+ 1065us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:50:40.094430 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:50:40.094431 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:50:40.094435 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:50:40.094449 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:50:40.094452 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:50:40.094502 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:40.094604 (+ 102us) server_negotiation.cc:299] Negotiation successful
0806 22:50:40.094619 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 22:50:40.094871 7767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:40.092545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:38049 (local address 127.28.218.194:37641)
0806 22:50:40.092736 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:40.092766 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:40.093196 (+ 430us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:40.093440 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:40.093444 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:40.093458 (+ 14us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:40.094279 (+ 821us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:50:40.094280 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:40.094303 (+ 23us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:50:40.094327 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:50:40.094499 (+ 172us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:50:40.094503 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:50:40.094506 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:50:40.094613 (+ 107us) client_negotiation.cc:240] Negotiation successful
0806 22:50:40.094670 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":95,"threads_started":1}
I20260806 22:50:40.095358 7765 heartbeater.cc:347] Connected to a master server at 127.28.218.254:38049
I20260806 22:50:40.095428 7765 heartbeater.cc:464] Registering TS with master...
I20260806 22:50:40.095584 7765 heartbeater.cc:511] Master 127.28.218.254:38049 requested a full tablet report, sending...
I20260806 22:50:40.096019 7432 ts_manager.cc:194] Registered new tserver with Master: b93627e1a7f446a1b54cd27824243576 (127.28.218.194:45263)
I20260806 22:50:40.098301 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 7629
I20260806 22:50:40.098407 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056586788339-29547-0/minicluster-data/ts-1/wal/instance
I20260806 22:50:40.098605 29547 external_mini_cluster.cc:1428] Setting key a588820532ef744c683dfeb1a8806974
I20260806 22:50:40.099439 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056586788339-29547-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056586788339-29547-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056586788339-29547-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056586788339-29547-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.218.195:0
--local_ip_for_outbound_sockets=127.28.218.195
--webserver_interface=127.28.218.195
--webserver_port=0
--tserver_master_addrs=127.28.218.254:38049
--builtin_ntp_servers=127.28.218.212:40399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260806 22:50:40.180169 7769 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:50:40.180361 7769 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:50:40.180382 7769 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:50:40.181449 7769 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:50:40.181484 7769 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:50:40.181509 7769 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.218.195
I20260806 22:50:40.182755 7769 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:40399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056586788339-29547-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056586788339-29547-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.218.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056586788339-29547-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.218.195
--webserver_port=0
--tserver_master_addrs=127.28.218.254:38049
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7769
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.218.195
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056586788339-29547-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:50:40.182921 7769 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:50:40.183159 7769 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:50:40.188122 7775 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:40.188126 7777 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:50:40.188181 7769 server_base.cc:1034] running on GCE node
W20260806 22:50:40.188100 7774 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:50:40.188544 7769 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:50:40.188733 7769 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:50:40.189868 7769 hybrid_clock.cc:648] HybridClock initialized: now 1786056640189848 us; error 38 us; skew 500 ppm
I20260806 22:50:40.191082 7769 webserver.cc:460] Webserver started at http://127.28.218.195:37243/ using document root <none> and password file <none>
I20260806 22:50:40.191324 7769 fs_manager.cc:359] Metadata directory not provided
I20260806 22:50:40.191370 7769 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:50:40.191457 7769 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:50:40.192456 7769 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056586788339-29547-0/minicluster-data/ts-2/data/instance:
uuid: "4f7dccb80f74490f86393adfb66ac103"
format_stamp: "Formatted at 2026-08-06 22:50:40 on dist-test-slave-1xrh"
server_key: "a70f93b238ff18c4863bf0e46c988e13"
server_key_iv: "ce86aa7def05ae2a363d4b3db732188a"
server_key_version: "encryptionkey@0"
I20260806 22:50:40.192756 7769 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056586788339-29547-0/minicluster-data/ts-2/wal/instance:
uuid: "4f7dccb80f74490f86393adfb66ac103"
format_stamp: "Formatted at 2026-08-06 22:50:40 on dist-test-slave-1xrh"
server_key: "a70f93b238ff18c4863bf0e46c988e13"
server_key_iv: "ce86aa7def05ae2a363d4b3db732188a"
server_key_version: "encryptionkey@0"
I20260806 22:50:40.193985 7769 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:50:40.194669 7784 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:40.194821 7769 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:50:40.194905 7769 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056586788339-29547-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056586788339-29547-0/minicluster-data/ts-2/wal
uuid: "4f7dccb80f74490f86393adfb66ac103"
format_stamp: "Formatted at 2026-08-06 22:50:40 on dist-test-slave-1xrh"
server_key: "a70f93b238ff18c4863bf0e46c988e13"
server_key_iv: "ce86aa7def05ae2a363d4b3db732188a"
server_key_version: "encryptionkey@0"
I20260806 22:50:40.194960 7769 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056586788339-29547-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056586788339-29547-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056586788339-29547-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:50:40.198189 7769 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:50:40.198382 7769 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:50:40.198493 7769 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:50:40.198648 7769 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:50:40.198879 7769 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:50:40.198907 7769 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:40.198927 7769 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:50:40.198939 7769 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:40.204491 7769 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.195:34869
I20260806 22:50:40.204567 7902 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.195:34869 every 8 connection(s)
I20260806 22:50:40.204864 7769 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056586788339-29547-0/minicluster-data/ts-2/data/info.pb
I20260806 22:50:40.207877 7472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:40.205574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.195:52869 (local address 127.28.218.254:38049)
0806 22:50:40.205641 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:40.205643 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:40.205888 (+ 245us) server_negotiation.cc:407] Connection header received
0806 22:50:40.206415 (+ 527us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:40.206417 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:40.206423 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:40.206486 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:40.207612 (+ 1126us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:50:40.207613 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:50:40.207613 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:50:40.207622 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:50:40.207636 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:50:40.207640 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:50:40.207699 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:40.207765 (+ 66us) server_negotiation.cc:299] Negotiation successful
0806 22:50:40.207778 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 22:50:40.207976 7905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:40.205535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:38049 (local address 127.28.218.195:52869)
0806 22:50:40.205740 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:40.205777 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:40.206239 (+ 462us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:40.206552 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:40.206555 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:40.206562 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:40.207475 (+ 913us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:50:40.207475 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:40.207500 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:50:40.207526 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:50:40.207700 (+ 174us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:50:40.207702 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:50:40.207704 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 22:50:40.207752 (+ 48us) client_negotiation.cc:240] Negotiation successful
0806 22:50:40.207797 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":93,"threads_started":1}
I20260806 22:50:40.208458 7903 heartbeater.cc:347] Connected to a master server at 127.28.218.254:38049
I20260806 22:50:40.208561 7903 heartbeater.cc:464] Registering TS with master...
I20260806 22:50:40.208716 7903 heartbeater.cc:511] Master 127.28.218.254:38049 requested a full tablet report, sending...
I20260806 22:50:40.209029 7432 ts_manager.cc:194] Registered new tserver with Master: 4f7dccb80f74490f86393adfb66ac103 (127.28.218.195:34869)
I20260806 22:50:40.214511 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 7769
I20260806 22:50:40.214644 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056586788339-29547-0/minicluster-data/ts-2/wal/instance
I20260806 22:50:40.214839 29547 external_mini_cluster.cc:1428] Setting key 8d25b99812d532eeac11dace46b2a439
I20260806 22:50:40.215585 29547 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 22:50:40.217305 7472 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:50:40.216804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45274 (local address 127.28.218.254:38049)
0806 22:50:40.216858 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:40.216859 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:40.216987 (+ 128us) server_negotiation.cc:407] Connection header received
0806 22:50:40.217079 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:40.217080 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:40.217085 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:40.217139 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:40.217244 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45274: BlockingRecv error: recv got EOF from 127.0.0.1:45274 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
I20260806 22:50:40.217384 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 7495
I20260806 22:50:40.224165 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 7629
I20260806 22:50:40.230036 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 7769
I20260806 22:50:40.234531 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 7399
2026-08-06T22:50:40Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (531 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056586788339-29547-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:50:40 dist-test-slave-1xrh krb5kdc[7914](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:50:40 dist-test-slave-1xrh krb5kdc[7914](info): set up 2 sockets
Aug 06 22:50:40 dist-test-slave-1xrh krb5kdc[7914](info): commencing operation
krb5kdc: starting...
W20260806 22:50:41.280855 29547 mini_kdc.cc:121] Time spent starting KDC: real 1.024s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056586788339-29547-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056586788339-29547-0/krb5kdc/test-user.keytab.
Aug 06 22:50:41 dist-test-slave-1xrh krb5kdc[7914](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056641, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:50:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:50:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.218.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.218.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.218.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab.
Entry for principal kudu/127.28.218.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab.
I20260806 22:50:41.406479 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056586788339-29547-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056586788339-29547-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056586788339-29547-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.218.254:34183
--webserver_interface=127.28.218.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab
--principal=kudu/127.28.218.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.218.212:43165
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.218.254:34183
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:50:41.486574 7928 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:50:41.486780 7928 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:50:41.486814 7928 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:50:41.487962 7928 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:50:41.488036 7928 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:50:41.488065 7928 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:50:41.488096 7928 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:50:41.488119 7928 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:50:41.489552 7928 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:43165
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056586788339-29547-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056586788339-29547-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.218.254:34183
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab
--principal=kudu/127.28.218.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.218.254:34183
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.218.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7928
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056586788339-29547-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:50:41.489821 7928 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:50:41.490042 7928 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:50:41.494652 7936 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:41.494701 7934 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:41.494664 7933 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:50:41.494900 7928 server_base.cc:1034] running on GCE node
I20260806 22:50:41.495189 7928 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:50:41.495468 7928 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:50:41.496629 7928 hybrid_clock.cc:648] HybridClock initialized: now 1786056641496621 us; error 38 us; skew 500 ppm
Aug 06 22:50:41 dist-test-slave-1xrh krb5kdc[7914](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056641, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:50:41.499086 7928 init.cc:377] Logged in from keytab as kudu/127.28.218.254@KRBTEST.COM (short username kudu)
I20260806 22:50:41.499557 7928 webserver.cc:460] Webserver started at http://127.28.218.254:36431/ using document root <none> and password file <none>
I20260806 22:50:41.499784 7928 fs_manager.cc:359] Metadata directory not provided
I20260806 22:50:41.499852 7928 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:50:41.500042 7928 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:50:41.501025 7928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056586788339-29547-0/minicluster-data/master-0/data/instance:
uuid: "99f6f04cf3f3461cb2d965fa9a1101f9"
format_stamp: "Formatted at 2026-08-06 22:50:41 on dist-test-slave-1xrh"
server_key: "926c598247cdfefd7f9f3215b0770ee9"
server_key_iv: "943f10bd318bb103f9809821ae71c550"
server_key_version: "encryptionkey@0"
I20260806 22:50:41.501398 7928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056586788339-29547-0/minicluster-data/master-0/wal/instance:
uuid: "99f6f04cf3f3461cb2d965fa9a1101f9"
format_stamp: "Formatted at 2026-08-06 22:50:41 on dist-test-slave-1xrh"
server_key: "926c598247cdfefd7f9f3215b0770ee9"
server_key_iv: "943f10bd318bb103f9809821ae71c550"
server_key_version: "encryptionkey@0"
I20260806 22:50:41.502667 7928 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:50:41.503427 7944 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:41.503595 7928 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:50:41.503693 7928 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056586788339-29547-0/minicluster-data/master-0/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056586788339-29547-0/minicluster-data/master-0/wal
uuid: "99f6f04cf3f3461cb2d965fa9a1101f9"
format_stamp: "Formatted at 2026-08-06 22:50:41 on dist-test-slave-1xrh"
server_key: "926c598247cdfefd7f9f3215b0770ee9"
server_key_iv: "943f10bd318bb103f9809821ae71c550"
server_key_version: "encryptionkey@0"
I20260806 22:50:41.503773 7928 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056586788339-29547-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056586788339-29547-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056586788339-29547-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:50:41.533951 7928 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:50:41.535794 7928 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:50:41.535969 7928 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:50:41.540117 7928 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.254:34183
I20260806 22:50:41.540153 7998 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.254:34183 every 8 connection(s)
I20260806 22:50:41.540520 7928 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
I20260806 22:50:41.541158 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 7928
I20260806 22:50:41.541286 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056586788339-29547-0/minicluster-data/master-0/wal/instance
I20260806 22:50:41.541512 29547 external_mini_cluster.cc:1428] Setting key b84673a86de7d4d755b5183f9a5d24c3
I20260806 22:50:41.541761 7999 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:50:41.545064 7999 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 99f6f04cf3f3461cb2d965fa9a1101f9: Bootstrap starting.
I20260806 22:50:41.545727 7999 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 99f6f04cf3f3461cb2d965fa9a1101f9: Neither blocks nor log segments found. Creating new log.
I20260806 22:50:41.546036 7999 log.cc:824] T 00000000000000000000000000000000 P 99f6f04cf3f3461cb2d965fa9a1101f9: Log is configured to *not* fsync() on all Append() calls
I20260806 22:50:41.547145 7999 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 99f6f04cf3f3461cb2d965fa9a1101f9: No bootstrap required, opened a new log
Aug 06 22:50:41 dist-test-slave-1xrh krb5kdc[7914](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056641, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.218.254@KRBTEST.COM
I20260806 22:50:41.549501 7999 raft_consensus.cc:348] T 00000000000000000000000000000000 P 99f6f04cf3f3461cb2d965fa9a1101f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99f6f04cf3f3461cb2d965fa9a1101f9" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 34183 } }
I20260806 22:50:41.549674 7999 raft_consensus.cc:374] T 00000000000000000000000000000000 P 99f6f04cf3f3461cb2d965fa9a1101f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:50:41.549717 7999 raft_consensus.cc:729] T 00000000000000000000000000000000 P 99f6f04cf3f3461cb2d965fa9a1101f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99f6f04cf3f3461cb2d965fa9a1101f9, State: Initialized, Role: FOLLOWER
I20260806 22:50:41.549906 7999 consensus_queue.cc:260] T 00000000000000000000000000000000 P 99f6f04cf3f3461cb2d965fa9a1101f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99f6f04cf3f3461cb2d965fa9a1101f9" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 34183 } }
I20260806 22:50:41.550030 8004 raft_consensus.cc:482] T 00000000000000000000000000000000 P 99f6f04cf3f3461cb2d965fa9a1101f9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:50:41.550153 8004 raft_consensus.cc:504] T 00000000000000000000000000000000 P 99f6f04cf3f3461cb2d965fa9a1101f9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99f6f04cf3f3461cb2d965fa9a1101f9" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 34183 } }
I20260806 22:50:41.550009 7999 raft_consensus.cc:388] T 00000000000000000000000000000000 P 99f6f04cf3f3461cb2d965fa9a1101f9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:50:41.550307 8004 leader_election.cc:302] T 00000000000000000000000000000000 P 99f6f04cf3f3461cb2d965fa9a1101f9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 99f6f04cf3f3461cb2d965fa9a1101f9; no voters:
I20260806 22:50:41.550323 7999 raft_consensus.cc:482] T 00000000000000000000000000000000 P 99f6f04cf3f3461cb2d965fa9a1101f9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:50:41.550405 7999 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 99f6f04cf3f3461cb2d965fa9a1101f9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:50:41.550509 8004 leader_election.cc:288] T 00000000000000000000000000000000 P 99f6f04cf3f3461cb2d965fa9a1101f9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers
I20260806 22:50:41.551158 7999 raft_consensus.cc:504] T 00000000000000000000000000000000 P 99f6f04cf3f3461cb2d965fa9a1101f9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99f6f04cf3f3461cb2d965fa9a1101f9" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 34183 } }
I20260806 22:50:41.551249 7999 leader_election.cc:302] T 00000000000000000000000000000000 P 99f6f04cf3f3461cb2d965fa9a1101f9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 99f6f04cf3f3461cb2d965fa9a1101f9; no voters:
I20260806 22:50:41.551291 7999 leader_election.cc:288] T 00000000000000000000000000000000 P 99f6f04cf3f3461cb2d965fa9a1101f9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:50:41.551298 8005 raft_consensus.cc:2741] T 00000000000000000000000000000000 P 99f6f04cf3f3461cb2d965fa9a1101f9 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260806 22:50:41.551486 8006 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 99f6f04cf3f3461cb2d965fa9a1101f9 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:50:41.551539 7999 sys_catalog.cc:564] T 00000000000000000000000000000000 P 99f6f04cf3f3461cb2d965fa9a1101f9 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:50:41.551541 8006 raft_consensus.cc:686] T 00000000000000000000000000000000 P 99f6f04cf3f3461cb2d965fa9a1101f9 [term 1 LEADER]: Becoming Leader. State: Replica: 99f6f04cf3f3461cb2d965fa9a1101f9, State: Running, Role: LEADER
I20260806 22:50:41.551640 8006 consensus_queue.cc:237] T 00000000000000000000000000000000 P 99f6f04cf3f3461cb2d965fa9a1101f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99f6f04cf3f3461cb2d965fa9a1101f9" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 34183 } }
I20260806 22:50:41.551944 8006 sys_catalog.cc:455] T 00000000000000000000000000000000 P 99f6f04cf3f3461cb2d965fa9a1101f9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 99f6f04cf3f3461cb2d965fa9a1101f9. Latest consensus state: current_term: 1 leader_uuid: "99f6f04cf3f3461cb2d965fa9a1101f9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99f6f04cf3f3461cb2d965fa9a1101f9" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 34183 } } }
I20260806 22:50:41.552187 8006 sys_catalog.cc:458] T 00000000000000000000000000000000 P 99f6f04cf3f3461cb2d965fa9a1101f9 [sys.catalog]: This master's current role is: LEADER
I20260806 22:50:41.552106 8005 sys_catalog.cc:455] T 00000000000000000000000000000000 P 99f6f04cf3f3461cb2d965fa9a1101f9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "99f6f04cf3f3461cb2d965fa9a1101f9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99f6f04cf3f3461cb2d965fa9a1101f9" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 34183 } } }
I20260806 22:50:41.552611 8005 sys_catalog.cc:458] T 00000000000000000000000000000000 P 99f6f04cf3f3461cb2d965fa9a1101f9 [sys.catalog]: This master's current role is: LEADER
I20260806 22:50:41.552927 8002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:41.542099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45372 (local address 127.28.218.254:34183)
0806 22:50:41.542268 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:41.542277 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:41.542311 (+ 34us) server_negotiation.cc:407] Connection header received
0806 22:50:41.542859 (+ 548us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:41.542865 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:41.542909 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:41.543019 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:41.543851 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:41.544713 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:41.545723 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:41.546202 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:41.546305 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:41.548854 (+ 2549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:50:41.548860 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:50:41.548861 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:50:41.548873 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:50:41.550857 (+ 1984us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:41.551222 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:41.551225 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:41.551225 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:41.551332 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:41.551608 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:41.551610 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:41.551610 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:41.551881 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:50:41.552058 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:41.552572 (+ 514us) server_negotiation.cc:299] Negotiation successful
0806 22:50:41.552714 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":90,"threads_started":1}
W20260806 22:50:41.553351 8020 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 99f6f04cf3f3461cb2d965fa9a1101f9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 22:50:41.553402 8020 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 22:50:41.553509 8013 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:50:41.553732 8013 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:50:41.555070 8013 catalog_manager.cc:1347] Generated new cluster ID: 2a542ec285884938b6d3e8e6a2643d55
I20260806 22:50:41.555176 8013 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:50:41.562031 8013 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:50:41.562546 8013 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:50:41.567584 8013 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 99f6f04cf3f3461cb2d965fa9a1101f9: Generated new TSK 0
I20260806 22:50:41.567756 8013 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.28.218.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.218.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.218.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.193.keytab.
Entry for principal kudu/127.28.218.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.193.keytab.
I20260806 22:50:41.597922 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056586788339-29547-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056586788339-29547-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056586788339-29547-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056586788339-29547-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.218.193:0
--local_ip_for_outbound_sockets=127.28.218.193
--webserver_interface=127.28.218.193
--webserver_port=0
--tserver_master_addrs=127.28.218.254:34183
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.193.keytab
--principal=kudu/127.28.218.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.218.212:43165
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:50:41.677783 8025 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:50:41.677969 8025 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:50:41.677991 8025 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:50:41.679178 8025 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:50:41.679219 8025 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:50:41.679265 8025 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.218.193
I20260806 22:50:41.680667 8025 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:43165
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056586788339-29547-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056586788339-29547-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.193.keytab
--principal=kudu/127.28.218.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.218.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056586788339-29547-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.218.193
--webserver_port=0
--tserver_master_addrs=127.28.218.254:34183
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8025
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.218.193
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056586788339-29547-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:50:41.680840 8025 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:50:41.681083 8025 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 22:50:41.686170 8025 server_base.cc:1034] running on GCE node
W20260806 22:50:41.686157 8033 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:41.686149 8031 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:41.686179 8030 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:50:41.686621 8025 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:50:41.686848 8025 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:50:41.688009 8025 hybrid_clock.cc:648] HybridClock initialized: now 1786056641688010 us; error 42 us; skew 500 ppm
Aug 06 22:50:41 dist-test-slave-1xrh krb5kdc[7914](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056641, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:50:41.690487 8025 init.cc:377] Logged in from keytab as kudu/127.28.218.193@KRBTEST.COM (short username kudu)
I20260806 22:50:41.690923 8025 webserver.cc:460] Webserver started at http://127.28.218.193:44401/ using document root <none> and password file <none>
I20260806 22:50:41.691181 8025 fs_manager.cc:359] Metadata directory not provided
I20260806 22:50:41.691251 8025 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:50:41.691383 8025 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:50:41.692482 8025 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056586788339-29547-0/minicluster-data/ts-0/data/instance:
uuid: "ed9d15516c474edfb5b769c211cfb8ff"
format_stamp: "Formatted at 2026-08-06 22:50:41 on dist-test-slave-1xrh"
server_key: "1f3cde5a8f91cfb4eac1523de728fea6"
server_key_iv: "324025bd7e528b39aa42521be2dfa61f"
server_key_version: "encryptionkey@0"
I20260806 22:50:41.692834 8025 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056586788339-29547-0/minicluster-data/ts-0/wal/instance:
uuid: "ed9d15516c474edfb5b769c211cfb8ff"
format_stamp: "Formatted at 2026-08-06 22:50:41 on dist-test-slave-1xrh"
server_key: "1f3cde5a8f91cfb4eac1523de728fea6"
server_key_iv: "324025bd7e528b39aa42521be2dfa61f"
server_key_version: "encryptionkey@0"
I20260806 22:50:41.694121 8025 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:50:41.694809 8040 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:41.694984 8025 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:50:41.695086 8025 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056586788339-29547-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056586788339-29547-0/minicluster-data/ts-0/wal
uuid: "ed9d15516c474edfb5b769c211cfb8ff"
format_stamp: "Formatted at 2026-08-06 22:50:41 on dist-test-slave-1xrh"
server_key: "1f3cde5a8f91cfb4eac1523de728fea6"
server_key_iv: "324025bd7e528b39aa42521be2dfa61f"
server_key_version: "encryptionkey@0"
I20260806 22:50:41.695191 8025 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056586788339-29547-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056586788339-29547-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056586788339-29547-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:50:41.705077 8025 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:50:41.706871 8025 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:50:41.707023 8025 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:50:41.707293 8025 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:50:41.707604 8025 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:50:41.707656 8025 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:41.707690 8025 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:50:41.707716 8025 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:41.713595 8025 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.193:35455
I20260806 22:50:41.713645 8167 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.193:35455 every 8 connection(s)
I20260806 22:50:41.713936 8025 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056586788339-29547-0/minicluster-data/ts-0/data/info.pb
Aug 06 22:50:41 dist-test-slave-1xrh krb5kdc[7914](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056641, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.193@KRBTEST.COM for kudu/127.28.218.254@KRBTEST.COM
I20260806 22:50:41.722337 8002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:41.714588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.193:40627 (local address 127.28.218.254:34183)
0806 22:50:41.714660 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:41.714662 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:41.714837 (+ 175us) server_negotiation.cc:407] Connection header received
0806 22:50:41.715319 (+ 482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:41.715320 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:41.715330 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:41.715378 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:41.716242 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:41.716740 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:41.717653 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:41.717846 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:41.717896 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:41.720056 (+ 2160us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:50:41.720058 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:50:41.720058 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:50:41.720065 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:50:41.721237 (+ 1172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:41.721595 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:41.721596 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:41.721596 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:41.721643 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:41.721845 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:41.721846 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:41.721846 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:41.721941 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:50:41.721990 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:41.722209 (+ 219us) server_negotiation.cc:299] Negotiation successful
0806 22:50:41.722272 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 22:50:41.722530 8170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:41.714545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:34183 (local address 127.28.218.193:40627)
0806 22:50:41.714725 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:41.714759 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:41.715223 (+ 464us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:41.715436 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:41.715439 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:41.715990 (+ 551us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:41.716170 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:41.716175 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:41.716816 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:41.716816 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:41.717580 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:41.717582 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:41.717927 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:41.717928 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:41.718027 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:41.718559 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:50:41.718568 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:41.719960 (+ 1392us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:50:41.721305 (+ 1345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:41.721312 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:41.721313 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:41.721533 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:41.721700 (+ 167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:41.721700 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:41.721701 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:41.721793 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:41.721995 (+ 202us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:50:41.721999 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:50:41.722103 (+ 104us) client_negotiation.cc:769] Sending connection context
0806 22:50:41.722182 (+ 79us) client_negotiation.cc:240] Negotiation successful
0806 22:50:41.722295 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":94,"threads_started":1}
I20260806 22:50:41.722888 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 8025
I20260806 22:50:41.723019 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056586788339-29547-0/minicluster-data/ts-0/wal/instance
I20260806 22:50:41.723272 29547 external_mini_cluster.cc:1428] Setting key 3516f470a5bbe59ec0eb7817cd02d48c
I20260806 22:50:41.723398 8168 heartbeater.cc:347] Connected to a master server at 127.28.218.254:34183
I20260806 22:50:41.723503 8168 heartbeater.cc:464] Registering TS with master...
I20260806 22:50:41.723716 8168 heartbeater.cc:511] Master 127.28.218.254:34183 requested a full tablet report, sending...
I20260806 22:50:41.724298 7961 ts_manager.cc:194] Registered new tserver with Master: ed9d15516c474edfb5b769c211cfb8ff (127.28.218.193:35455)
I20260806 22:50:41.724891 7961 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.218.193@KRBTEST.COM'} at 127.28.218.193:40627
WARNING: no policy specified for kudu/127.28.218.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.218.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.218.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.194.keytab.
Entry for principal kudu/127.28.218.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.194.keytab.
I20260806 22:50:41.749835 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056586788339-29547-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056586788339-29547-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056586788339-29547-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056586788339-29547-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.218.194:0
--local_ip_for_outbound_sockets=127.28.218.194
--webserver_interface=127.28.218.194
--webserver_port=0
--tserver_master_addrs=127.28.218.254:34183
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.194.keytab
--principal=kudu/127.28.218.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.218.212:43165
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:50:41.829125 8173 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:50:41.829313 8173 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:50:41.829335 8173 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:50:41.830488 8173 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:50:41.830535 8173 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:50:41.830591 8173 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.218.194
I20260806 22:50:41.832013 8173 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:43165
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056586788339-29547-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056586788339-29547-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.194.keytab
--principal=kudu/127.28.218.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.218.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056586788339-29547-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.218.194
--webserver_port=0
--tserver_master_addrs=127.28.218.254:34183
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8173
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.218.194
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056586788339-29547-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:50:41.832239 8173 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:50:41.832453 8173 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:50:41.837397 8178 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:41.837388 8179 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:50:41.837420 8173 server_base.cc:1034] running on GCE node
W20260806 22:50:41.837389 8181 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:50:41.837935 8173 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:50:41.838130 8173 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:50:41.839303 8173 hybrid_clock.cc:648] HybridClock initialized: now 1786056641839284 us; error 30 us; skew 500 ppm
Aug 06 22:50:41 dist-test-slave-1xrh krb5kdc[7914](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056641, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:50:41.841648 8173 init.cc:377] Logged in from keytab as kudu/127.28.218.194@KRBTEST.COM (short username kudu)
I20260806 22:50:41.842098 8173 webserver.cc:460] Webserver started at http://127.28.218.194:37025/ using document root <none> and password file <none>
I20260806 22:50:41.842265 8173 fs_manager.cc:359] Metadata directory not provided
I20260806 22:50:41.842303 8173 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:50:41.842383 8173 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:50:41.843355 8173 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056586788339-29547-0/minicluster-data/ts-1/data/instance:
uuid: "c290cac0085c4065ac4c1b7837fb4081"
format_stamp: "Formatted at 2026-08-06 22:50:41 on dist-test-slave-1xrh"
server_key: "04f858f9353848a7228948f6d47b4c9d"
server_key_iv: "76c70391be86d593ae5b92eba40727fa"
server_key_version: "encryptionkey@0"
I20260806 22:50:41.843641 8173 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056586788339-29547-0/minicluster-data/ts-1/wal/instance:
uuid: "c290cac0085c4065ac4c1b7837fb4081"
format_stamp: "Formatted at 2026-08-06 22:50:41 on dist-test-slave-1xrh"
server_key: "04f858f9353848a7228948f6d47b4c9d"
server_key_iv: "76c70391be86d593ae5b92eba40727fa"
server_key_version: "encryptionkey@0"
I20260806 22:50:41.844852 8173 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:50:41.845481 8188 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:41.845654 8173 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:50:41.845758 8173 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056586788339-29547-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056586788339-29547-0/minicluster-data/ts-1/wal
uuid: "c290cac0085c4065ac4c1b7837fb4081"
format_stamp: "Formatted at 2026-08-06 22:50:41 on dist-test-slave-1xrh"
server_key: "04f858f9353848a7228948f6d47b4c9d"
server_key_iv: "76c70391be86d593ae5b92eba40727fa"
server_key_version: "encryptionkey@0"
I20260806 22:50:41.845827 8173 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056586788339-29547-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056586788339-29547-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056586788339-29547-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:50:41.856971 8173 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:50:41.858654 8173 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:50:41.858804 8173 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:50:41.859011 8173 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:50:41.859326 8173 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:50:41.859360 8173 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:41.859380 8173 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:50:41.859428 8173 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:41.865284 8173 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.194:42687
I20260806 22:50:41.865334 8307 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.194:42687 every 8 connection(s)
I20260806 22:50:41.865614 8173 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056586788339-29547-0/minicluster-data/ts-1/data/info.pb
Aug 06 22:50:41 dist-test-slave-1xrh krb5kdc[7914](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056641, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.194@KRBTEST.COM for kudu/127.28.218.254@KRBTEST.COM
I20260806 22:50:41.874114 8002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:41.866291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.194:53987 (local address 127.28.218.254:34183)
0806 22:50:41.866367 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:41.866369 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:41.866537 (+ 168us) server_negotiation.cc:407] Connection header received
0806 22:50:41.866994 (+ 457us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:41.866995 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:41.867006 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:41.867061 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:41.868008 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:41.868499 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:41.869415 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:41.869596 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:41.869645 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:41.871878 (+ 2233us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:50:41.871880 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:50:41.871880 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:50:41.871886 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:50:41.873037 (+ 1151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:41.873375 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:41.873376 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:41.873376 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:41.873423 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:41.873623 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:41.873623 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:41.873623 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:41.873714 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:50:41.873763 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:41.873981 (+ 218us) server_negotiation.cc:299] Negotiation successful
0806 22:50:41.874050 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 22:50:41.874287 8310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:41.866241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:34183 (local address 127.28.218.194:53987)
0806 22:50:41.866420 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:41.866454 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:41.866913 (+ 459us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:41.867164 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:41.867168 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:41.867740 (+ 572us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:41.867932 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:41.867936 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:41.868577 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:41.868577 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:41.869345 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:41.869347 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:41.869666 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:41.869667 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:41.869760 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:41.870277 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:50:41.870285 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:41.871794 (+ 1509us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:50:41.873106 (+ 1312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:41.873111 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:41.873113 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:41.873313 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:41.873480 (+ 167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:41.873480 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:41.873480 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:41.873571 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:41.873769 (+ 198us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:50:41.873773 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:50:41.873873 (+ 100us) client_negotiation.cc:769] Sending connection context
0806 22:50:41.873956 (+ 83us) client_negotiation.cc:240] Negotiation successful
0806 22:50:41.874068 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":91,"threads_started":1}
I20260806 22:50:41.874852 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 8173
I20260806 22:50:41.874876 8308 heartbeater.cc:347] Connected to a master server at 127.28.218.254:34183
I20260806 22:50:41.874962 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056586788339-29547-0/minicluster-data/ts-1/wal/instance
I20260806 22:50:41.874976 8308 heartbeater.cc:464] Registering TS with master...
I20260806 22:50:41.875182 8308 heartbeater.cc:511] Master 127.28.218.254:34183 requested a full tablet report, sending...
I20260806 22:50:41.875223 29547 external_mini_cluster.cc:1428] Setting key 2ed272d31f12628d08a362dcfe5166b7
I20260806 22:50:41.875564 7961 ts_manager.cc:194] Registered new tserver with Master: c290cac0085c4065ac4c1b7837fb4081 (127.28.218.194:42687)
I20260806 22:50:41.876017 7961 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.218.194@KRBTEST.COM'} at 127.28.218.194:53987
WARNING: no policy specified for kudu/127.28.218.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.218.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.218.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.195.keytab.
Entry for principal kudu/127.28.218.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.195.keytab.
I20260806 22:50:41.901167 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056586788339-29547-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056586788339-29547-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056586788339-29547-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056586788339-29547-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.218.195:0
--local_ip_for_outbound_sockets=127.28.218.195
--webserver_interface=127.28.218.195
--webserver_port=0
--tserver_master_addrs=127.28.218.254:34183
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.195.keytab
--principal=kudu/127.28.218.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.218.212:43165
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:50:41.981297 8314 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:50:41.981500 8314 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:50:41.981523 8314 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:50:41.982589 8314 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:50:41.982620 8314 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:50:41.982646 8314 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.218.195
I20260806 22:50:41.984033 8314 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:43165
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056586788339-29547-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056586788339-29547-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.195.keytab
--principal=kudu/127.28.218.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.218.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056586788339-29547-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.218.195
--webserver_port=0
--tserver_master_addrs=127.28.218.254:34183
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8314
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.218.195
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056586788339-29547-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:50:41.984211 8314 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:50:41.984377 8314 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:50:41.989223 8322 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:41.989302 8320 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:50:41.989249 8319 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:50:41.989629 8314 server_base.cc:1034] running on GCE node
I20260806 22:50:41.989790 8314 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:50:41.989972 8314 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:50:41.991276 8314 hybrid_clock.cc:648] HybridClock initialized: now 1786056641991245 us; error 35 us; skew 500 ppm
Aug 06 22:50:41 dist-test-slave-1xrh krb5kdc[7914](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056641, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:50:41.993700 8314 init.cc:377] Logged in from keytab as kudu/127.28.218.195@KRBTEST.COM (short username kudu)
I20260806 22:50:41.994150 8314 webserver.cc:460] Webserver started at http://127.28.218.195:32935/ using document root <none> and password file <none>
I20260806 22:50:41.994367 8314 fs_manager.cc:359] Metadata directory not provided
I20260806 22:50:41.994433 8314 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:50:41.994588 8314 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:50:41.995653 8314 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056586788339-29547-0/minicluster-data/ts-2/data/instance:
uuid: "eb48636e3b2343cd8c907de7a3903989"
format_stamp: "Formatted at 2026-08-06 22:50:41 on dist-test-slave-1xrh"
server_key: "a177a346c33907ad01abcfa1dd3ea503"
server_key_iv: "d7e399d9a017ceab5953c332a7c74c1e"
server_key_version: "encryptionkey@0"
I20260806 22:50:41.996021 8314 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056586788339-29547-0/minicluster-data/ts-2/wal/instance:
uuid: "eb48636e3b2343cd8c907de7a3903989"
format_stamp: "Formatted at 2026-08-06 22:50:41 on dist-test-slave-1xrh"
server_key: "a177a346c33907ad01abcfa1dd3ea503"
server_key_iv: "d7e399d9a017ceab5953c332a7c74c1e"
server_key_version: "encryptionkey@0"
I20260806 22:50:41.997400 8314 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:50:41.998150 8329 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:41.998332 8314 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:50:41.998430 8314 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056586788339-29547-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056586788339-29547-0/minicluster-data/ts-2/wal
uuid: "eb48636e3b2343cd8c907de7a3903989"
format_stamp: "Formatted at 2026-08-06 22:50:41 on dist-test-slave-1xrh"
server_key: "a177a346c33907ad01abcfa1dd3ea503"
server_key_iv: "d7e399d9a017ceab5953c332a7c74c1e"
server_key_version: "encryptionkey@0"
I20260806 22:50:41.998521 8314 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056586788339-29547-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056586788339-29547-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056586788339-29547-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:50:42.019599 8314 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:50:42.021445 8314 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:50:42.021636 8314 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:50:42.021883 8314 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:50:42.022204 8314 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:50:42.022257 8314 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:42.022293 8314 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:50:42.022341 8314 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:50:42.028164 8314 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.195:43759
I20260806 22:50:42.028219 8445 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.195:43759 every 8 connection(s)
I20260806 22:50:42.028533 8314 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056586788339-29547-0/minicluster-data/ts-2/data/info.pb
Aug 06 22:50:42 dist-test-slave-1xrh krb5kdc[7914](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056641, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.195@KRBTEST.COM for kudu/127.28.218.254@KRBTEST.COM
I20260806 22:50:42.035856 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 8314
I20260806 22:50:42.035946 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056586788339-29547-0/minicluster-data/ts-2/wal/instance
I20260806 22:50:42.036182 29547 external_mini_cluster.cc:1428] Setting key 8b5d896ce9132d872b81e58bf7148f29
I20260806 22:50:42.037814 8002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:42.029277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.195:32967 (local address 127.28.218.254:34183)
0806 22:50:42.029323 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:42.029325 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:42.029488 (+ 163us) server_negotiation.cc:407] Connection header received
0806 22:50:42.029964 (+ 476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:42.029965 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:42.029974 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:42.030030 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:42.030921 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:42.031498 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:42.032423 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:42.032612 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:42.032668 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:42.034712 (+ 2044us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:50:42.034713 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:50:42.034714 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:50:42.034720 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:50:42.036423 (+ 1703us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:42.036935 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:42.036937 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:42.036937 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:42.037003 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:42.037294 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:42.037295 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:42.037295 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:42.037397 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:50:42.037454 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:42.037683 (+ 229us) server_negotiation.cc:299] Negotiation successful
0806 22:50:42.037751 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 22:50:42.037987 8448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:42.029207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:34183 (local address 127.28.218.195:32967)
0806 22:50:42.029359 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:42.029400 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:42.029872 (+ 472us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:42.030083 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:42.030086 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:42.030658 (+ 572us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:50:42.030842 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:42.030847 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:42.031580 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:42.031581 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:42.032344 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:42.032346 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:42.032685 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:42.032685 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:42.032780 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:42.033194 (+ 414us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:50:42.033202 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:50:42.034603 (+ 1401us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:50:42.036503 (+ 1900us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:42.036510 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:42.036518 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:42.036864 (+ 346us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:42.037072 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:50:42.037073 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:50:42.037074 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:50:42.037229 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:50:42.037457 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:50:42.037461 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:50:42.037574 (+ 113us) client_negotiation.cc:769] Sending connection context
0806 22:50:42.037656 (+ 82us) client_negotiation.cc:240] Negotiation successful
0806 22:50:42.037768 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":87,"threads_started":1}
I20260806 22:50:42.038400 8446 heartbeater.cc:347] Connected to a master server at 127.28.218.254:34183
I20260806 22:50:42.038492 8446 heartbeater.cc:464] Registering TS with master...
I20260806 22:50:42.038672 8446 heartbeater.cc:511] Master 127.28.218.254:34183 requested a full tablet report, sending...
I20260806 22:50:42.039044 7961 ts_manager.cc:194] Registered new tserver with Master: eb48636e3b2343cd8c907de7a3903989 (127.28.218.195:43759)
I20260806 22:50:42.039449 7961 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.218.195@KRBTEST.COM'} at 127.28.218.195:32967
I20260806 22:50:42.047873 29547 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:50:42.055267 8002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:42.049320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45376 (local address 127.28.218.254:34183)
0806 22:50:42.049416 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:42.049418 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:42.049472 (+ 54us) server_negotiation.cc:407] Connection header received
0806 22:50:42.049526 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:42.049527 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:42.049535 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:42.049588 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:50:42.050126 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:42.050594 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:42.051359 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:42.051546 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:42.051597 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:42.052875 (+ 1278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:50:42.052876 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:50:42.052876 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:50:42.052883 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:50:42.054287 (+ 1404us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:42.054581 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:42.054582 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:42.054582 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:42.054630 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:50:42.054786 (+ 156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:50:42.054787 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:50:42.054787 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:50:42.054930 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:50:42.055004 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:42.055091 (+ 87us) server_negotiation.cc:299] Negotiation successful
0806 22:50:42.055197 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 22:50:42.056761 7961 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45376:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:50:42.058399 7961 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:50:42.064673 8455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:42.060456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.195:43759 (local address 127.0.0.1:41808)
0806 22:50:42.060907 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:42.060915 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:42.060983 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:42.061295 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:42.061296 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:42.061298 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:50:42.061344 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:42.061346 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:42.062603 (+ 1257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:42.062604 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:42.063534 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:42.063537 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:42.064414 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:42.064415 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:42.064468 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:42.064479 (+ 11us) client_negotiation.cc:769] Sending connection context
0806 22:50:42.064538 (+ 59us) client_negotiation.cc:240] Negotiation successful
0806 22:50:42.064601 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":67,"threads_started":1}
I20260806 22:50:42.064697 8458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:42.060676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41808 (local address 127.28.218.195:43759)
0806 22:50:42.060852 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:42.060857 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:42.060979 (+ 122us) server_negotiation.cc:407] Connection header received
0806 22:50:42.061056 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:42.061058 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:42.061073 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:42.061400 (+ 327us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:50:42.061471 (+ 71us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:42.062505 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:42.063614 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:42.064338 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:42.064419 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:42.064472 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:42.064566 (+ 94us) server_negotiation.cc:299] Negotiation successful
0806 22:50:42.064609 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":91,"threads_started":1}
I20260806 22:50:42.064851 8454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:42.060444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.193:35455 (local address 127.0.0.1:36318)
0806 22:50:42.060544 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:42.060585 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:42.060649 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:42.061080 (+ 431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:42.061084 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:42.061087 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:50:42.061150 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:42.061152 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:42.062462 (+ 1310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:42.062462 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:42.063721 (+ 1259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:42.063724 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:42.064611 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:42.064611 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:42.064668 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:42.064671 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:50:42.064723 (+ 52us) client_negotiation.cc:240] Negotiation successful
0806 22:50:42.064769 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":77,"mutex_wait_us":53,"thread_start_us":57,"threads_started":1}
I20260806 22:50:42.065070 8457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:42.060613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36318 (local address 127.28.218.193:35455)
0806 22:50:42.060890 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:42.060894 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:42.060903 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:50:42.060935 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:42.060936 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:42.060949 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:42.061072 (+ 123us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:50:42.061225 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:42.062347 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:42.063817 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:42.064532 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:42.064601 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:42.064649 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:42.064911 (+ 262us) server_negotiation.cc:299] Negotiation successful
0806 22:50:42.064954 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":232,"thread_start_us":92,"threads_started":1}
I20260806 22:50:42.065748 8456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:42.060454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.194:42687 (local address 127.0.0.1:32826)
0806 22:50:42.060789 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:42.060800 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:42.060846 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:42.061452 (+ 606us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:42.061453 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:42.061455 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:50:42.061491 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:42.061492 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:42.062345 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:42.062346 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:42.063721 (+ 1375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:42.063724 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:42.065483 (+ 1759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:42.065484 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:42.065556 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:42.065560 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:50:42.065628 (+ 68us) client_negotiation.cc:240] Negotiation successful
0806 22:50:42.065673 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":321,"mutex_wait_us":34,"thread_start_us":54,"threads_started":1}
I20260806 22:50:42.065747 8092 tablet_service.cc:1467] Processing CreateTablet for tablet be402f44690d4c088025ee10054f2873 (DEFAULT_TABLE table=test-table [id=fa61bd940cc54e27b92fb9486bb634c9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:50:42.065747 8377 tablet_service.cc:1467] Processing CreateTablet for tablet be402f44690d4c088025ee10054f2873 (DEFAULT_TABLE table=test-table [id=fa61bd940cc54e27b92fb9486bb634c9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:50:42.066071 8459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:42.060680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32826 (local address 127.28.218.194:42687)
0806 22:50:42.061235 (+ 555us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:42.061239 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:42.061248 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:50:42.061283 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:42.061285 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:42.061298 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:42.061370 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:50:42.061584 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:42.062244 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:42.064684 (+ 2440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:42.065333 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:42.065392 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:42.065449 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:42.065944 (+ 495us) server_negotiation.cc:299] Negotiation successful
0806 22:50:42.065989 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":518,"thread_start_us":89,"threads_started":1}
I20260806 22:50:42.066087 8092 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be402f44690d4c088025ee10054f2873. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:50:42.066085 8377 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be402f44690d4c088025ee10054f2873. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:50:42.067678 8238 tablet_service.cc:1467] Processing CreateTablet for tablet be402f44690d4c088025ee10054f2873 (DEFAULT_TABLE table=test-table [id=fa61bd940cc54e27b92fb9486bb634c9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:50:42.068015 8238 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be402f44690d4c088025ee10054f2873. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:50:42.069401 8460 tablet_bootstrap.cc:492] T be402f44690d4c088025ee10054f2873 P eb48636e3b2343cd8c907de7a3903989: Bootstrap starting.
I20260806 22:50:42.070078 8460 tablet_bootstrap.cc:654] T be402f44690d4c088025ee10054f2873 P eb48636e3b2343cd8c907de7a3903989: Neither blocks nor log segments found. Creating new log.
I20260806 22:50:42.070662 8460 log.cc:824] T be402f44690d4c088025ee10054f2873 P eb48636e3b2343cd8c907de7a3903989: Log is configured to *not* fsync() on all Append() calls
I20260806 22:50:42.071003 8461 tablet_bootstrap.cc:492] T be402f44690d4c088025ee10054f2873 P ed9d15516c474edfb5b769c211cfb8ff: Bootstrap starting.
I20260806 22:50:42.071077 8463 tablet_bootstrap.cc:492] T be402f44690d4c088025ee10054f2873 P c290cac0085c4065ac4c1b7837fb4081: Bootstrap starting.
I20260806 22:50:42.071605 8460 tablet_bootstrap.cc:492] T be402f44690d4c088025ee10054f2873 P eb48636e3b2343cd8c907de7a3903989: No bootstrap required, opened a new log
I20260806 22:50:42.071593 8461 tablet_bootstrap.cc:654] T be402f44690d4c088025ee10054f2873 P ed9d15516c474edfb5b769c211cfb8ff: Neither blocks nor log segments found. Creating new log.
I20260806 22:50:42.071724 8463 tablet_bootstrap.cc:654] T be402f44690d4c088025ee10054f2873 P c290cac0085c4065ac4c1b7837fb4081: Neither blocks nor log segments found. Creating new log.
I20260806 22:50:42.071746 8460 ts_tablet_manager.cc:1397] T be402f44690d4c088025ee10054f2873 P eb48636e3b2343cd8c907de7a3903989: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:50:42.072180 8461 log.cc:824] T be402f44690d4c088025ee10054f2873 P ed9d15516c474edfb5b769c211cfb8ff: Log is configured to *not* fsync() on all Append() calls
I20260806 22:50:42.072314 8463 log.cc:824] T be402f44690d4c088025ee10054f2873 P c290cac0085c4065ac4c1b7837fb4081: Log is configured to *not* fsync() on all Append() calls
I20260806 22:50:42.073048 8461 tablet_bootstrap.cc:492] T be402f44690d4c088025ee10054f2873 P ed9d15516c474edfb5b769c211cfb8ff: No bootstrap required, opened a new log
I20260806 22:50:42.073048 8463 tablet_bootstrap.cc:492] T be402f44690d4c088025ee10054f2873 P c290cac0085c4065ac4c1b7837fb4081: No bootstrap required, opened a new log
I20260806 22:50:42.073120 8463 ts_tablet_manager.cc:1397] T be402f44690d4c088025ee10054f2873 P c290cac0085c4065ac4c1b7837fb4081: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:50:42.073120 8461 ts_tablet_manager.cc:1397] T be402f44690d4c088025ee10054f2873 P ed9d15516c474edfb5b769c211cfb8ff: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:50:42.073581 8460 raft_consensus.cc:348] T be402f44690d4c088025ee10054f2873 P eb48636e3b2343cd8c907de7a3903989 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed9d15516c474edfb5b769c211cfb8ff" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 35455 } } peers { permanent_uuid: "c290cac0085c4065ac4c1b7837fb4081" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 42687 } } peers { permanent_uuid: "eb48636e3b2343cd8c907de7a3903989" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 43759 } }
I20260806 22:50:42.073694 8460 raft_consensus.cc:374] T be402f44690d4c088025ee10054f2873 P eb48636e3b2343cd8c907de7a3903989 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:50:42.073714 8460 raft_consensus.cc:729] T be402f44690d4c088025ee10054f2873 P eb48636e3b2343cd8c907de7a3903989 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb48636e3b2343cd8c907de7a3903989, State: Initialized, Role: FOLLOWER
I20260806 22:50:42.073823 8460 consensus_queue.cc:260] T be402f44690d4c088025ee10054f2873 P eb48636e3b2343cd8c907de7a3903989 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed9d15516c474edfb5b769c211cfb8ff" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 35455 } } peers { permanent_uuid: "c290cac0085c4065ac4c1b7837fb4081" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 42687 } } peers { permanent_uuid: "eb48636e3b2343cd8c907de7a3903989" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 43759 } }
I20260806 22:50:42.074069 8446 heartbeater.cc:503] Master 127.28.218.254:34183 was elected leader, sending a full tablet report...
I20260806 22:50:42.074080 8460 ts_tablet_manager.cc:1428] T be402f44690d4c088025ee10054f2873 P eb48636e3b2343cd8c907de7a3903989: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:50:42.075023 8463 raft_consensus.cc:348] T be402f44690d4c088025ee10054f2873 P c290cac0085c4065ac4c1b7837fb4081 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed9d15516c474edfb5b769c211cfb8ff" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 35455 } } peers { permanent_uuid: "c290cac0085c4065ac4c1b7837fb4081" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 42687 } } peers { permanent_uuid: "eb48636e3b2343cd8c907de7a3903989" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 43759 } }
I20260806 22:50:42.075023 8461 raft_consensus.cc:348] T be402f44690d4c088025ee10054f2873 P ed9d15516c474edfb5b769c211cfb8ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed9d15516c474edfb5b769c211cfb8ff" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 35455 } } peers { permanent_uuid: "c290cac0085c4065ac4c1b7837fb4081" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 42687 } } peers { permanent_uuid: "eb48636e3b2343cd8c907de7a3903989" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 43759 } }
I20260806 22:50:42.075206 8461 raft_consensus.cc:374] T be402f44690d4c088025ee10054f2873 P ed9d15516c474edfb5b769c211cfb8ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:50:42.075206 8463 raft_consensus.cc:374] T be402f44690d4c088025ee10054f2873 P c290cac0085c4065ac4c1b7837fb4081 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:50:42.075244 8461 raft_consensus.cc:729] T be402f44690d4c088025ee10054f2873 P ed9d15516c474edfb5b769c211cfb8ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed9d15516c474edfb5b769c211cfb8ff, State: Initialized, Role: FOLLOWER
I20260806 22:50:42.075244 8463 raft_consensus.cc:729] T be402f44690d4c088025ee10054f2873 P c290cac0085c4065ac4c1b7837fb4081 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c290cac0085c4065ac4c1b7837fb4081, State: Initialized, Role: FOLLOWER
I20260806 22:50:42.075352 8461 consensus_queue.cc:260] T be402f44690d4c088025ee10054f2873 P ed9d15516c474edfb5b769c211cfb8ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed9d15516c474edfb5b769c211cfb8ff" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 35455 } } peers { permanent_uuid: "c290cac0085c4065ac4c1b7837fb4081" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 42687 } } peers { permanent_uuid: "eb48636e3b2343cd8c907de7a3903989" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 43759 } }
I20260806 22:50:42.075352 8463 consensus_queue.cc:260] T be402f44690d4c088025ee10054f2873 P c290cac0085c4065ac4c1b7837fb4081 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed9d15516c474edfb5b769c211cfb8ff" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 35455 } } peers { permanent_uuid: "c290cac0085c4065ac4c1b7837fb4081" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 42687 } } peers { permanent_uuid: "eb48636e3b2343cd8c907de7a3903989" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 43759 } }
I20260806 22:50:42.075623 8461 ts_tablet_manager.cc:1428] T be402f44690d4c088025ee10054f2873 P ed9d15516c474edfb5b769c211cfb8ff: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 22:50:42.075623 8463 ts_tablet_manager.cc:1428] T be402f44690d4c088025ee10054f2873 P c290cac0085c4065ac4c1b7837fb4081: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:50:42.075642 8308 heartbeater.cc:503] Master 127.28.218.254:34183 was elected leader, sending a full tablet report...
I20260806 22:50:42.075744 8168 heartbeater.cc:503] Master 127.28.218.254:34183 was elected leader, sending a full tablet report...
W20260806 22:50:42.116186 8309 tablet.cc:2369] T be402f44690d4c088025ee10054f2873 P c290cac0085c4065ac4c1b7837fb4081: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:50:42.187917 8467 raft_consensus.cc:482] T be402f44690d4c088025ee10054f2873 P ed9d15516c474edfb5b769c211cfb8ff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:50:42.188094 8467 raft_consensus.cc:504] T be402f44690d4c088025ee10054f2873 P ed9d15516c474edfb5b769c211cfb8ff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed9d15516c474edfb5b769c211cfb8ff" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 35455 } } peers { permanent_uuid: "c290cac0085c4065ac4c1b7837fb4081" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 42687 } } peers { permanent_uuid: "eb48636e3b2343cd8c907de7a3903989" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 43759 } }
I20260806 22:50:42.188470 8467 leader_election.cc:288] T be402f44690d4c088025ee10054f2873 P ed9d15516c474edfb5b769c211cfb8ff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c290cac0085c4065ac4c1b7837fb4081 (127.28.218.194:42687), eb48636e3b2343cd8c907de7a3903989 (127.28.218.195:43759)
I20260806 22:50:42.192569 8458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:42.188750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.193:53043 (local address 127.28.218.195:43759)
0806 22:50:42.188795 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:42.188797 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:42.188849 (+ 52us) server_negotiation.cc:407] Connection header received
0806 22:50:42.188873 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:42.188874 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:42.188881 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:42.188936 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:50:42.189122 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:42.189884 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:42.191600 (+ 1716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:42.192204 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:42.192277 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:42.192286 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:42.192464 (+ 178us) server_negotiation.cc:299] Negotiation successful
0806 22:50:42.192501 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 22:50:42.192569 8459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:42.188745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.193:52037 (local address 127.28.218.194:42687)
0806 22:50:42.188802 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:42.188804 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:42.188990 (+ 186us) server_negotiation.cc:407] Connection header received
0806 22:50:42.189031 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:42.189032 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:42.189041 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:42.189108 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:50:42.189231 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:42.189917 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:42.191596 (+ 1679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:42.192195 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:42.192277 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:42.192286 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:42.192464 (+ 178us) server_negotiation.cc:299] Negotiation successful
0806 22:50:42.192501 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 22:50:42.192581 8170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:42.188662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.195:43759 (local address 127.28.218.193:53043)
0806 22:50:42.188734 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:42.188747 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:42.188793 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:42.188956 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:42.188957 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:42.188959 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:50:42.189039 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:42.189041 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:42.190013 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:42.190014 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:42.191511 (+ 1497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:42.191513 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:42.192297 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:42.192297 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:42.192390 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:42.192392 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 22:50:42.192444 (+ 52us) client_negotiation.cc:240] Negotiation successful
0806 22:50:42.192490 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49,"mutex_wait_us":97}
I20260806 22:50:42.192588 8469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:42.188662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.194:42687 (local address 127.28.218.193:52037)
0806 22:50:42.188851 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:42.188860 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:50:42.188978 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:50:42.189109 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:50:42.189110 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:50:42.189111 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:50:42.189164 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:42.189165 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:42.190013 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:42.190014 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:42.191511 (+ 1497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:50:42.191513 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:50:42.192297 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:50:42.192298 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:50:42.192382 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:42.192386 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:50:42.192444 (+ 58us) client_negotiation.cc:240] Negotiation successful
0806 22:50:42.192490 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"mutex_wait_us":90,"thread_start_us":106,"threads_started":1}
I20260806 22:50:42.192978 8399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be402f44690d4c088025ee10054f2873" candidate_uuid: "ed9d15516c474edfb5b769c211cfb8ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb48636e3b2343cd8c907de7a3903989" is_pre_election: true
I20260806 22:50:42.193049 8260 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be402f44690d4c088025ee10054f2873" candidate_uuid: "ed9d15516c474edfb5b769c211cfb8ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c290cac0085c4065ac4c1b7837fb4081" is_pre_election: true
I20260806 22:50:42.193159 8399 raft_consensus.cc:2454] T be402f44690d4c088025ee10054f2873 P eb48636e3b2343cd8c907de7a3903989 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ed9d15516c474edfb5b769c211cfb8ff in term 0.
I20260806 22:50:42.193197 8260 raft_consensus.cc:2454] T be402f44690d4c088025ee10054f2873 P c290cac0085c4065ac4c1b7837fb4081 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ed9d15516c474edfb5b769c211cfb8ff in term 0.
I20260806 22:50:42.193427 8043 leader_election.cc:302] T be402f44690d4c088025ee10054f2873 P ed9d15516c474edfb5b769c211cfb8ff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: eb48636e3b2343cd8c907de7a3903989, ed9d15516c474edfb5b769c211cfb8ff; no voters:
I20260806 22:50:42.193578 8467 raft_consensus.cc:2781] T be402f44690d4c088025ee10054f2873 P ed9d15516c474edfb5b769c211cfb8ff [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:50:42.193636 8467 raft_consensus.cc:482] T be402f44690d4c088025ee10054f2873 P ed9d15516c474edfb5b769c211cfb8ff [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:50:42.193661 8467 raft_consensus.cc:3037] T be402f44690d4c088025ee10054f2873 P ed9d15516c474edfb5b769c211cfb8ff [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:50:42.194358 8467 raft_consensus.cc:504] T be402f44690d4c088025ee10054f2873 P ed9d15516c474edfb5b769c211cfb8ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed9d15516c474edfb5b769c211cfb8ff" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 35455 } } peers { permanent_uuid: "c290cac0085c4065ac4c1b7837fb4081" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 42687 } } peers { permanent_uuid: "eb48636e3b2343cd8c907de7a3903989" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 43759 } }
I20260806 22:50:42.194495 8467 leader_election.cc:288] T be402f44690d4c088025ee10054f2873 P ed9d15516c474edfb5b769c211cfb8ff [CANDIDATE]: Term 1 election: Requested vote from peers c290cac0085c4065ac4c1b7837fb4081 (127.28.218.194:42687), eb48636e3b2343cd8c907de7a3903989 (127.28.218.195:43759)
I20260806 22:50:42.194666 8399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be402f44690d4c088025ee10054f2873" candidate_uuid: "ed9d15516c474edfb5b769c211cfb8ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb48636e3b2343cd8c907de7a3903989"
I20260806 22:50:42.194648 8260 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be402f44690d4c088025ee10054f2873" candidate_uuid: "ed9d15516c474edfb5b769c211cfb8ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c290cac0085c4065ac4c1b7837fb4081"
I20260806 22:50:42.194739 8260 raft_consensus.cc:3037] T be402f44690d4c088025ee10054f2873 P c290cac0085c4065ac4c1b7837fb4081 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:50:42.194739 8399 raft_consensus.cc:3037] T be402f44690d4c088025ee10054f2873 P eb48636e3b2343cd8c907de7a3903989 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:50:42.195684 8399 raft_consensus.cc:2454] T be402f44690d4c088025ee10054f2873 P eb48636e3b2343cd8c907de7a3903989 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed9d15516c474edfb5b769c211cfb8ff in term 1.
I20260806 22:50:42.195684 8260 raft_consensus.cc:2454] T be402f44690d4c088025ee10054f2873 P c290cac0085c4065ac4c1b7837fb4081 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed9d15516c474edfb5b769c211cfb8ff in term 1.
I20260806 22:50:42.195924 8043 leader_election.cc:302] T be402f44690d4c088025ee10054f2873 P ed9d15516c474edfb5b769c211cfb8ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: eb48636e3b2343cd8c907de7a3903989, ed9d15516c474edfb5b769c211cfb8ff; no voters:
I20260806 22:50:42.196058 8467 raft_consensus.cc:2781] T be402f44690d4c088025ee10054f2873 P ed9d15516c474edfb5b769c211cfb8ff [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:50:42.196192 8467 raft_consensus.cc:686] T be402f44690d4c088025ee10054f2873 P ed9d15516c474edfb5b769c211cfb8ff [term 1 LEADER]: Becoming Leader. State: Replica: ed9d15516c474edfb5b769c211cfb8ff, State: Running, Role: LEADER
I20260806 22:50:42.196293 8467 consensus_queue.cc:237] T be402f44690d4c088025ee10054f2873 P ed9d15516c474edfb5b769c211cfb8ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed9d15516c474edfb5b769c211cfb8ff" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 35455 } } peers { permanent_uuid: "c290cac0085c4065ac4c1b7837fb4081" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 42687 } } peers { permanent_uuid: "eb48636e3b2343cd8c907de7a3903989" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 43759 } }
I20260806 22:50:42.197211 7961 catalog_manager.cc:5495] T be402f44690d4c088025ee10054f2873 P ed9d15516c474edfb5b769c211cfb8ff reported cstate change: term changed from 0 to 1, leader changed from <none> to ed9d15516c474edfb5b769c211cfb8ff (127.28.218.193). New cstate: current_term: 1 leader_uuid: "ed9d15516c474edfb5b769c211cfb8ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed9d15516c474edfb5b769c211cfb8ff" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 35455 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c290cac0085c4065ac4c1b7837fb4081" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 42687 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb48636e3b2343cd8c907de7a3903989" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 43759 } health_report { overall_health: UNKNOWN } } }
I20260806 22:50:42.207989 8457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:42.205720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36326 (local address 127.28.218.193:35455)
0806 22:50:42.205777 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0806 22:50:42.205778 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:50:42.205785 (+ 7us) server_negotiation.cc:407] Connection header received
0806 22:50:42.205810 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:50:42.205811 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:50:42.205826 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:50:42.205893 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:50:42.205998 (+ 105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:42.206487 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:42.207332 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:50:42.207551 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:50:42.207607 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:50:42.207719 (+ 112us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:50:42.207829 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:50:42.207892 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 22:50:42.207911 (+ 19us) server_negotiation.cc:299] Negotiation successful
0806 22:50:42.207927 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260806 22:50:42.209658 8260 raft_consensus.cc:1261] T be402f44690d4c088025ee10054f2873 P c290cac0085c4065ac4c1b7837fb4081 [term 1 FOLLOWER]: Refusing update from remote peer ed9d15516c474edfb5b769c211cfb8ff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:50:42.209658 8399 raft_consensus.cc:1261] T be402f44690d4c088025ee10054f2873 P eb48636e3b2343cd8c907de7a3903989 [term 1 FOLLOWER]: Refusing update from remote peer ed9d15516c474edfb5b769c211cfb8ff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:50:42.209987 8467 consensus_queue.cc:1035] T be402f44690d4c088025ee10054f2873 P ed9d15516c474edfb5b769c211cfb8ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb48636e3b2343cd8c907de7a3903989" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 43759 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:50:42.210170 8470 consensus_queue.cc:1035] T be402f44690d4c088025ee10054f2873 P ed9d15516c474edfb5b769c211cfb8ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "c290cac0085c4065ac4c1b7837fb4081" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 42687 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:50:42.214643 7962 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45376:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:50:42.214751 7962 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45376:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:50:42.215456 7962 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 99f6f04cf3f3461cb2d965fa9a1101f9: Sending DeleteTablet for 3 replicas of tablet be402f44690d4c088025ee10054f2873
I20260806 22:50:42.215794 8377 tablet_service.cc:1514] Processing DeleteTablet for tablet be402f44690d4c088025ee10054f2873 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:50:42 UTC) from {username='kudu', principal='kudu/127.28.218.254@KRBTEST.COM'} at 127.0.0.1:41808
I20260806 22:50:42.215837 8238 tablet_service.cc:1514] Processing DeleteTablet for tablet be402f44690d4c088025ee10054f2873 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:50:42 UTC) from {username='kudu', principal='kudu/127.28.218.254@KRBTEST.COM'} at 127.0.0.1:32826
I20260806 22:50:42.215965 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 8025
I20260806 22:50:42.217093 8484 tablet_replica.cc:330] T be402f44690d4c088025ee10054f2873 P eb48636e3b2343cd8c907de7a3903989: stopping tablet replica
I20260806 22:50:42.217206 8484 raft_consensus.cc:2229] T be402f44690d4c088025ee10054f2873 P eb48636e3b2343cd8c907de7a3903989 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:50:42.217316 8484 raft_consensus.cc:2258] T be402f44690d4c088025ee10054f2873 P eb48636e3b2343cd8c907de7a3903989 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:50:42.217696 8484 ts_tablet_manager.cc:1905] T be402f44690d4c088025ee10054f2873 P eb48636e3b2343cd8c907de7a3903989: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:50:42.217916 8485 tablet_replica.cc:330] T be402f44690d4c088025ee10054f2873 P c290cac0085c4065ac4c1b7837fb4081: stopping tablet replica
I20260806 22:50:42.218144 8485 raft_consensus.cc:2229] T be402f44690d4c088025ee10054f2873 P c290cac0085c4065ac4c1b7837fb4081 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:50:42.218261 8485 raft_consensus.cc:2258] T be402f44690d4c088025ee10054f2873 P c290cac0085c4065ac4c1b7837fb4081 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:50:42.218694 8485 ts_tablet_manager.cc:1905] T be402f44690d4c088025ee10054f2873 P c290cac0085c4065ac4c1b7837fb4081: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:50:42.219529 8484 ts_tablet_manager.cc:1918] T be402f44690d4c088025ee10054f2873 P eb48636e3b2343cd8c907de7a3903989: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:50:42.219624 8484 log.cc:1196] T be402f44690d4c088025ee10054f2873 P eb48636e3b2343cd8c907de7a3903989: Deleting WAL directory at /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056586788339-29547-0/minicluster-data/ts-2/wal/wals/be402f44690d4c088025ee10054f2873
I20260806 22:50:42.219908 8484 ts_tablet_manager.cc:1939] T be402f44690d4c088025ee10054f2873 P eb48636e3b2343cd8c907de7a3903989: Deleting consensus metadata
I20260806 22:50:42.220304 7947 catalog_manager.cc:4841] TS eb48636e3b2343cd8c907de7a3903989 (127.28.218.195:43759): tablet be402f44690d4c088025ee10054f2873 (table test-table [id=fa61bd940cc54e27b92fb9486bb634c9]) successfully deleted
I20260806 22:50:42.220564 8485 ts_tablet_manager.cc:1918] T be402f44690d4c088025ee10054f2873 P c290cac0085c4065ac4c1b7837fb4081: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:50:42.220631 8485 log.cc:1196] T be402f44690d4c088025ee10054f2873 P c290cac0085c4065ac4c1b7837fb4081: Deleting WAL directory at /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056586788339-29547-0/minicluster-data/ts-1/wal/wals/be402f44690d4c088025ee10054f2873
I20260806 22:50:42.220973 8485 ts_tablet_manager.cc:1939] T be402f44690d4c088025ee10054f2873 P c290cac0085c4065ac4c1b7837fb4081: Deleting consensus metadata
I20260806 22:50:42.221374 7948 catalog_manager.cc:4841] TS c290cac0085c4065ac4c1b7837fb4081 (127.28.218.194:42687): tablet be402f44690d4c088025ee10054f2873 (table test-table [id=fa61bd940cc54e27b92fb9486bb634c9]) successfully deleted
W20260806 22:50:42.223532 7946 connection.cc:535] client connection to 127.28.218.193:35455 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260806 22:50:42.223667 7946 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 22:50:42.223829 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 8173
I20260806 22:50:42.226423 8455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:50:42.226278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.193:35455 (local address 127.0.0.1:36340)
0806 22:50:42.226334 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0806 22:50:42.226372 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.193:35455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260806 22:50:42.226568 7946 catalog_manager.cc:4584] TS ed9d15516c474edfb5b769c211cfb8ff (127.28.218.193:35455): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet be402f44690d4c088025ee10054f2873: Network error: Client connection negotiation failed: client connection to 127.28.218.193:35455: connect: Connection refused (error 111)
I20260806 22:50:42.229579 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 8314
I20260806 22:50:42.235524 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 7928
2026-08-06T22:50:42Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (1999 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:50:42 dist-test-slave-1xrh krb5kdc[8489](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:50:42 dist-test-slave-1xrh krb5kdc[8489](info): set up 2 sockets
Aug 06 22:50:42 dist-test-slave-1xrh krb5kdc[8489](info): commencing operation
krb5kdc: starting...
W20260806 22:50:43.295780 29547 mini_kdc.cc:121] Time spent starting KDC: real 1.039s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/krb5kdc/test-user.keytab.
Aug 06 22:50:43 dist-test-slave-1xrh krb5kdc[8489](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056643, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:50:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:50:43Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.28.218.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.28.218.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.28.218.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/krb5kdc/rangeradmin_127.28.218.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.28.218.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/krb5kdc/rangeradmin_127.28.218.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.28.218.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.28.218.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.28.218.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/krb5kdc/rangerlookup_127.28.218.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.28.218.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/krb5kdc/rangerlookup_127.28.218.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.28.218.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.28.218.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.28.218.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/krb5kdc/HTTP_127.28.218.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.28.218.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/krb5kdc/HTTP_127.28.218.212@KRBTEST.COM.keytab.
I20260806 22:50:43.461248 29547 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskQja9Us/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/minicluster-data/postgres -l logfile start
2026-08-06 22:50:45.967 UTC [8520] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-06 22:50:45.968 UTC [8520] LOG: listening on IPv4 address "127.28.218.212", port 59091
2026-08-06 22:50:45.971 UTC [8520] LOG: listening on Unix socket "/tmp/.s.PGSQL.59091"
2026-08-06 22:50:45.993 UTC [8523] LOG: database system was shut down at 2026-08-06 22:50:44 UTC
2026-08-06 22:50:45.997 UTC [8520] LOG: database system is ready to accept connections
I20260806 22:50:46.971544 29547 mini_postgres.cc:96] Postgres bound to 59091
2026-08-06 22:50:46.978 UTC [8531] FATAL: database "slave" does not exist
127.28.218.212:59091 - accepting connections
I20260806 22:50:46.979624 29547 mini_ranger.cc:161] Starting Ranger...
I20260806 22:50:46.994586 29547 mini_ranger.cc:84] Created miniranger Postgres user
I20260806 22:50:47.456812 29547 mini_ranger.cc:87] Created ranger Postgres database
I20260806 22:50:47.456897 29547 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/minicluster-data/ranger-admin
2026-08-06 22:50:47,754 [I] DB FLAVOR :POSTGRES
2026-08-06 22:50:47,754 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 22:50:47,754 [I] Checking connection..
2026-08-06 22:50:47,754 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQja9Us/build/release/bin/postgresql.jar:/tmp/dist-test-taskQja9Us/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.218.212:59091/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-06 22:50:48,009 [I] Checking connection passed.
2026-08-06 22:50:48,009 [I] --------- Verifying version history table ---------
2026-08-06 22:50:48,009 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQja9Us/build/release/bin/postgresql.jar:/tmp/dist-test-taskQja9Us/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.218.212:59091/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 22:50:48,265 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 22:50:48,266 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQja9Us/build/release/bin/postgresql.jar:/tmp/dist-test-taskQja9Us/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.218.212:59091/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 22:50:48,531 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 22:50:48,531 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-06 22:50:48,532 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQja9Us/build/release/bin/postgresql.jar:/tmp/dist-test-taskQja9Us/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.218.212:59091/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskQja9Us/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-06 22:50:48.785 UTC [8600] WARNING: there is no transaction in progress
2026-08-06 22:50:48,792 [I] create_dbversion_catalog.sql file imported successfully
2026-08-06 22:50:48,792 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQja9Us/build/release/bin/postgresql.jar:/tmp/dist-test-taskQja9Us/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.218.212:59091/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 22:50:49,034 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-06 22:50:49,034 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 22:50:49,034 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQja9Us/build/release/bin/postgresql.jar:/tmp/dist-test-taskQja9Us/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.218.212:59091/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-06 22:50:49,289 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQja9Us/build/release/bin/postgresql.jar:/tmp/dist-test-taskQja9Us/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.218.212:59091/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-06 22:50:49,544 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQja9Us/build/release/bin/postgresql.jar:/tmp/dist-test-taskQja9Us/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.218.212:59091/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-1xrh.c.gcp-upstream.internal','N') ;"
2026-08-06 22:50:49,762 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQja9Us/build/release/bin/postgresql.jar:/tmp/dist-test-taskQja9Us/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.218.212:59091/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 22:50:49,986 [I] Table x_portal_user does not exist in database ranger
2026-08-06 22:50:49,987 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQja9Us/build/release/bin/postgresql.jar:/tmp/dist-test-taskQja9Us/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.218.212:59091/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 22:50:50,236 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-06 22:50:50,237 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQja9Us/build/release/bin/postgresql.jar:/tmp/dist-test-taskQja9Us/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.218.212:59091/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 22:50:50,483 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQja9Us/build/release/bin/postgresql.jar:/tmp/dist-test-taskQja9Us/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.218.212:59091/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 22:50:50,727 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-06 22:50:50,727 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQja9Us/build/release/bin/postgresql.jar:/tmp/dist-test-taskQja9Us/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.218.212:59091/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskQja9Us/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-06 22:50:51.663 UTC [8716] WARNING: there is no transaction in progress
2026-08-06 22:50:51.679 UTC [8716] WARNING: there is no transaction in progress
2026-08-06 22:50:51.695 UTC [8716] WARNING: there is no transaction in progress
2026-08-06 22:50:51.710 UTC [8716] WARNING: there is no transaction in progress
2026-08-06 22:50:51.729 UTC [8716] WARNING: there is no transaction in progress
2026-08-06 22:50:51.748 UTC [8716] WARNING: there is no transaction in progress
2026-08-06 22:50:51.811 UTC [8716] WARNING: there is no transaction in progress
2026-08-06 22:50:51.818 UTC [8716] WARNING: there is no transaction in progress
2026-08-06 22:50:51.832 UTC [8716] WARNING: there is no transaction in progress
2026-08-06 22:50:51.842 UTC [8716] WARNING: there is no transaction in progress
2026-08-06 22:50:51.851 UTC [8716] WARNING: there is no transaction in progress
2026-08-06 22:50:51.864 UTC [8716] WARNING: there is no transaction in progress
2026-08-06 22:50:51.873 UTC [8716] WARNING: there is no transaction in progress
2026-08-06 22:50:51.879 UTC [8716] WARNING: there is no transaction in progress
2026-08-06 22:50:51.886 UTC [8716] WARNING: there is no transaction in progress
2026-08-06 22:50:51.892 UTC [8716] WARNING: there is no transaction in progress
2026-08-06 22:50:52,517 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-06 22:50:52,517 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQja9Us/build/release/bin/postgresql.jar:/tmp/dist-test-taskQja9Us/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.218.212:59091/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-06 22:50:52,733 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-06 22:50:52,734 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQja9Us/build/release/bin/postgresql.jar:/tmp/dist-test-taskQja9Us/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.218.212:59091/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 22:50:52,966 [I] Table x_portal_user already exists in database 'ranger'
2026-08-06 22:50:52,966 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQja9Us/build/release/bin/postgresql.jar:/tmp/dist-test-taskQja9Us/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.218.212:59091/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 22:50:53,217 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-06 22:50:53,217 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQja9Us/build/release/bin/postgresql.jar:/tmp/dist-test-taskQja9Us/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.218.212:59091/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 22:50:53,444 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQja9Us/build/release/bin/postgresql.jar:/tmp/dist-test-taskQja9Us/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.218.212:59091/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 22:50:53,671 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQja9Us/build/release/bin/postgresql.jar:/tmp/dist-test-taskQja9Us/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.218.212:59091/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-1xrh.c.gcp-upstream.internal';"
2026-08-06 22:50:53,913 [I] CORE_DB_SCHEMA import status has been updated
2026-08-06 22:50:53,913 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQja9Us/build/release/bin/postgresql.jar:/tmp/dist-test-taskQja9Us/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.218.212:59091/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-06 22:50:54,167 [I] DB_PATCHES have already been applied
I20260806 22:50:54.174299 29547 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/minicluster-data/ranger-admin:/tmp/dist-test-taskQja9Us/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskQja9Us/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskQja9Us/build/release/bin/postgresql.jar:/tmp/dist-test-taskQja9Us/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260806 22:50:54.174389 29547 mini_ranger.cc:193] Using host: 127.28.218.212
I20260806 22:50:54.178190 29547 mini_ranger.cc:227] Ranger admin URL: http://127.28.218.212:58251
Aug 06, 2026 10:50:54 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 10:50:54 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 10:50:54 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/minicluster-data/ranger-admin/ews/webapp
Aug 06, 2026 10:50:54 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 06, 2026 10:50:54 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 06, 2026 10:50:55 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/minicluster-data/ranger-admin/ews/webapp].
Aug 06, 2026 10:50:55 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260806 22:50:55.312740 29547 mini_ranger.cc:160] Time spent starting Ranger: real 8.333s user 0.002s sys 0.008s
Aug 06, 2026 10:51:14 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 06, 2026 10:51:14 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 06, 2026 10:51:14 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 06, 2026 10:51:14 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 06, 2026 10:51:14 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 06, 2026 10:51:14 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 06, 2026 10:51:14 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 06, 2026 10:51:14 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 06, 2026 10:51:14 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 06, 2026 10:51:14 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 06, 2026 10:51:14 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 06, 2026 10:51:14 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 06, 2026 10:51:14 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 06, 2026 10:51:14 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 06, 2026 10:51:14 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 06, 2026 10:51:14 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 06, 2026 10:51:14 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 06, 2026 10:51:15 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260806 22:51:16.446714 29547 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.28.218.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.28.218.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.28.218.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/krb5kdc/rangerkms_127.28.218.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.28.218.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/krb5kdc/rangerkms_127.28.218.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.28.218.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.28.218.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.28.218.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/krb5kdc/HTTP_127.28.218.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.28.218.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/krb5kdc/HTTP_127.28.218.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 06 22:51:16 dist-test-slave-1xrh krb5kdc[8489](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056676, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260806 22:51:16.549060 29547 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260806 22:51:16.559584 29547 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260806 22:51:17.408030 29547 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260806 22:51:17.408120 29547 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/minicluster-data/ranger-kms
I20260806 22:51:17.408142 29547 mini_ranger_kms.cc:227] Using postgres at 127.28.218.212:59091
2026-08-06 22:51:17,823 [I] DB FLAVOR :POSTGRES
2026-08-06 22:51:17,823 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 22:51:17,824 [I] Checking connection
2026-08-06 22:51:17,824 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQja9Us/build/release/bin/postgresql.jar:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.218.212:59091/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-06 22:51:18,140 [I] connection success
2026-08-06 22:51:18,140 [I] --------- Verifying Ranger DB tables ---------
2026-08-06 22:51:18,140 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-06 22:51:18,141 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQja9Us/build/release/bin/postgresql.jar:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.218.212:59091/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-06 22:51:18,380 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-06 22:51:18,381 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 22:51:18,381 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-06 22:51:18,381 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQja9Us/build/release/bin/postgresql.jar:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.218.212:59091/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-06 22:51:18,647 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260806 22:51:19.140980 29547 mini_ranger_kms.cc:310] Created kms service
I20260806 22:51:19.329826 29547 mini_ranger_kms.cc:326] Created kudu user
I20260806 22:51:19.375300 29547 mini_ranger_kms.cc:343] Created rangerkms user
Aug 06 22:51:19 dist-test-slave-1xrh krb5kdc[8489](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056676, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.28.218.212@KRBTEST.COM
I20260806 22:51:19.777562 29547 mini_ranger_kms.cc:384] Added ranger policy
I20260806 22:51:19.777678 29547 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/minicluster-data/ranger-kms:/tmp/dist-test-taskQja9Us/build/release/bin/postgresql.jar:/tmp/dist-test-taskQja9Us/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskQja9Us/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260806 22:51:19.777699 29547 mini_ranger_kms.cc:240] Using host: 127.28.218.212
I20260806 22:51:19.782096 29547 mini_ranger_kms.cc:276] Ranger KMS PID: 8917
I20260806 22:51:19.782209 29547 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.28.218.212:52807
Aug 06, 2026 10:51:21 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 10:51:21 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 10:51:21 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 06, 2026 10:51:21 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 06, 2026 10:51:21 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/minicluster-data/ranger-kms/ews/webapp].
Aug 06, 2026 10:51:21 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260806 22:51:22.153820 29547 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.605s user 0.016s sys 0.377s
I20260806 22:51:22.153939 29547 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260806 22:51:22.154023 29547 mini_ranger_kms.cc:401] 127.28.218.212:52807/kms/v1/keys
Aug 06 22:51:22 dist-test-slave-1xrh krb5kdc[8489](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786056682, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.28.218.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 06 22:51:22 dist-test-slave-1xrh krb5kdc[8489](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056682, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.28.218.212@KRBTEST.COM for HTTP/127.28.218.212@KRBTEST.COM
Aug 06 22:51:23 dist-test-slave-1xrh krb5kdc[8489](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056682, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.28.218.212@KRBTEST.COM for HTTP/127.28.218.212@KRBTEST.COM
Aug 06 22:51:25 dist-test-slave-1xrh krb5kdc[8489](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056685, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.28.218.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.218.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.218.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab.
Entry for principal kudu/127.28.218.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab.
I20260806 22:51:25.564124 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.218.254:36251
--webserver_interface=127.28.218.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab
--principal=kudu/127.28.218.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.218.212:46329
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.218.254:36251
--ranger_config_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.28.218.212:52807/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:51:25.679992 8979 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:51:25.680187 8979 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:51:25.680208 8979 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:51:25.681308 8979 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:51:25.681344 8979 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:51:25.681358 8979 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:51:25.681370 8979 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:51:25.681382 8979 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:51:25.682953 8979 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:46329
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/minicluster-data/master-0/wal
--ranger_kms_url=127.28.218.212:52807/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.28.218.254:36251
--ranger_config_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab
--principal=kudu/127.28.218.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.218.254:36251
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.218.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8979
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:51:25.683198 8979 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:51:25.683378 8979 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:51:25.688920 8985 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:51:25.689077 8986 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:51:25.688962 8979 server_base.cc:1034] running on GCE node
W20260806 22:51:25.689082 8988 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:51:25.689448 8979 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:51:25.689736 8979 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:51:25.690915 8979 hybrid_clock.cc:648] HybridClock initialized: now 1786056685690900 us; error 46 us; skew 500 ppm
Aug 06 22:51:25 dist-test-slave-1xrh krb5kdc[8489](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056685, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:51:25.693678 8979 init.cc:377] Logged in from keytab as kudu/127.28.218.254@KRBTEST.COM (short username kudu)
I20260806 22:51:25.694140 8979 webserver.cc:460] Webserver started at http://127.28.218.254:39493/ using document root <none> and password file <none>
I20260806 22:51:25.694356 8979 fs_manager.cc:359] Metadata directory not provided
I20260806 22:51:25.694415 8979 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:51:25.694573 8979 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 22:51:25 dist-test-slave-1xrh krb5kdc[8489](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056685, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.254@KRBTEST.COM for HTTP/127.28.218.212@KRBTEST.COM
I20260806 22:51:25.742432 8979 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/minicluster-data/master-0/data/instance:
uuid: "0eb6e1a9c254404888f746c6b82dca6f"
format_stamp: "Formatted at 2026-08-06 22:51:25 on dist-test-slave-1xrh"
server_key: "1367024d2758e30df8b4ff99f0d37190"
server_key_iv: "a5f032018fe527faa54bbcd13632eae3"
server_key_version: "kuduclusterkey@0"
I20260806 22:51:25.742910 8979 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/minicluster-data/master-0/wal/instance:
uuid: "0eb6e1a9c254404888f746c6b82dca6f"
format_stamp: "Formatted at 2026-08-06 22:51:25 on dist-test-slave-1xrh"
server_key: "1367024d2758e30df8b4ff99f0d37190"
server_key_iv: "a5f032018fe527faa54bbcd13632eae3"
server_key_version: "kuduclusterkey@0"
I20260806 22:51:25.744889 8979 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.001s
I20260806 22:51:25.759850 8996 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:51:25.760129 8979 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 22:51:25.760303 8979 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/minicluster-data/master-0/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/minicluster-data/master-0/wal
uuid: "0eb6e1a9c254404888f746c6b82dca6f"
format_stamp: "Formatted at 2026-08-06 22:51:25 on dist-test-slave-1xrh"
server_key: "1367024d2758e30df8b4ff99f0d37190"
server_key_iv: "a5f032018fe527faa54bbcd13632eae3"
server_key_version: "kuduclusterkey@0"
I20260806 22:51:25.760406 8979 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:51:25.783418 8979 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:51:25.785703 8979 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:51:25.785859 8979 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:51:25.794517 8979 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.254:36251
I20260806 22:51:25.794559 9055 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.254:36251 every 8 connection(s)
I20260806 22:51:25.795053 8979 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
I20260806 22:51:25.795951 9056 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:51:25.799708 9056 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0eb6e1a9c254404888f746c6b82dca6f: Bootstrap starting.
I20260806 22:51:25.800714 9056 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0eb6e1a9c254404888f746c6b82dca6f: Neither blocks nor log segments found. Creating new log.
I20260806 22:51:25.800953 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 8979
I20260806 22:51:25.801074 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/minicluster-data/master-0/wal/instance
I20260806 22:51:25.801174 9056 log.cc:824] T 00000000000000000000000000000000 P 0eb6e1a9c254404888f746c6b82dca6f: Log is configured to *not* fsync() on all Append() calls
I20260806 22:51:25.801328 29547 external_mini_cluster.cc:1428] Setting key 394d28670d72c927d29ed5b3daf95bba
I20260806 22:51:25.802114 9056 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0eb6e1a9c254404888f746c6b82dca6f: No bootstrap required, opened a new log
I20260806 22:51:25.806257 9056 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0eb6e1a9c254404888f746c6b82dca6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0eb6e1a9c254404888f746c6b82dca6f" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 36251 } }
I20260806 22:51:25.806440 9056 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0eb6e1a9c254404888f746c6b82dca6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:51:25.806473 9056 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0eb6e1a9c254404888f746c6b82dca6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0eb6e1a9c254404888f746c6b82dca6f, State: Initialized, Role: FOLLOWER
I20260806 22:51:25.806615 9056 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0eb6e1a9c254404888f746c6b82dca6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0eb6e1a9c254404888f746c6b82dca6f" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 36251 } }
I20260806 22:51:25.806685 9056 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0eb6e1a9c254404888f746c6b82dca6f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:51:25.806715 9056 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0eb6e1a9c254404888f746c6b82dca6f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:51:25.806778 9056 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0eb6e1a9c254404888f746c6b82dca6f [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:51:25.807614 9056 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0eb6e1a9c254404888f746c6b82dca6f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0eb6e1a9c254404888f746c6b82dca6f" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 36251 } }
I20260806 22:51:25.807762 9056 leader_election.cc:302] T 00000000000000000000000000000000 P 0eb6e1a9c254404888f746c6b82dca6f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0eb6e1a9c254404888f746c6b82dca6f; no voters:
I20260806 22:51:25.807936 9056 leader_election.cc:288] T 00000000000000000000000000000000 P 0eb6e1a9c254404888f746c6b82dca6f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:51:25.807988 9062 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0eb6e1a9c254404888f746c6b82dca6f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:51:25.808110 9062 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0eb6e1a9c254404888f746c6b82dca6f [term 1 LEADER]: Becoming Leader. State: Replica: 0eb6e1a9c254404888f746c6b82dca6f, State: Running, Role: LEADER
I20260806 22:51:25.808284 9056 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0eb6e1a9c254404888f746c6b82dca6f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:51:25.809577 9062 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0eb6e1a9c254404888f746c6b82dca6f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0eb6e1a9c254404888f746c6b82dca6f" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 36251 } }
Aug 06 22:51:25 dist-test-slave-1xrh krb5kdc[8489](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056685, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.218.254@KRBTEST.COM
I20260806 22:51:25.810240 9064 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0eb6e1a9c254404888f746c6b82dca6f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0eb6e1a9c254404888f746c6b82dca6f. Latest consensus state: current_term: 1 leader_uuid: "0eb6e1a9c254404888f746c6b82dca6f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0eb6e1a9c254404888f746c6b82dca6f" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 36251 } } }
I20260806 22:51:25.810325 9064 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0eb6e1a9c254404888f746c6b82dca6f [sys.catalog]: This master's current role is: LEADER
I20260806 22:51:25.810595 9063 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0eb6e1a9c254404888f746c6b82dca6f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0eb6e1a9c254404888f746c6b82dca6f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0eb6e1a9c254404888f746c6b82dca6f" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 36251 } } }
I20260806 22:51:25.810662 9063 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0eb6e1a9c254404888f746c6b82dca6f [sys.catalog]: This master's current role is: LEADER
I20260806 22:51:25.810896 9074 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:51:25.811509 9074 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:51:25.813001 9074 catalog_manager.cc:1347] Generated new cluster ID: fafbce4e042d4315b769f594e8331652
I20260806 22:51:25.813058 9074 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:51:25.813277 9056 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260806 22:51:25.816051 9061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:25.801910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60242 (local address 127.28.218.254:36251)
0806 22:51:25.802154 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:25.802164 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:25.802237 (+ 73us) server_negotiation.cc:407] Connection header received
0806 22:51:25.802819 (+ 582us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:25.802829 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:25.802892 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:25.803091 (+ 199us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:51:25.804160 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:25.805812 (+ 1652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:25.806908 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:25.807553 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:25.807658 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:25.811220 (+ 3562us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:51:25.811225 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:51:25.811226 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:51:25.811239 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:51:25.814418 (+ 3179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:51:25.814802 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:51:25.814805 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:51:25.814805 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:51:25.814902 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:51:25.815108 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:51:25.815109 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:51:25.815109 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:51:25.815428 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:51:25.815611 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:25.815680 (+ 69us) server_negotiation.cc:299] Negotiation successful
0806 22:51:25.815792 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":189,"thread_start_us":155,"threads_started":1}
I20260806 22:51:25.838346 9074 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:51:25.838985 9074 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:51:25.849691 9074 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0eb6e1a9c254404888f746c6b82dca6f: Generated new TSK 0
I20260806 22:51:25.849907 9074 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.28.218.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.218.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.218.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.193.keytab.
Entry for principal kudu/127.28.218.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.193.keytab.
I20260806 22:51:26.622574 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.28.218.212:52807/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.218.193:0
--local_ip_for_outbound_sockets=127.28.218.193
--webserver_interface=127.28.218.193
--webserver_port=0
--tserver_master_addrs=127.28.218.254:36251
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.193.keytab
--principal=kudu/127.28.218.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.218.212:46329
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:51:26.741204 9105 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:51:26.741436 9105 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:51:26.741459 9105 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:51:26.742698 9105 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:51:26.742740 9105 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:51:26.742765 9105 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.218.193
I20260806 22:51:26.744299 9105 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:46329
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.28.218.212:52807/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.193.keytab
--principal=kudu/127.28.218.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.218.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.218.193
--webserver_port=0
--tserver_master_addrs=127.28.218.254:36251
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9105
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.218.193
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:51:26.744525 9105 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:51:26.744786 9105 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:51:26.750204 9112 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:51:26.750334 9111 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:51:26.750204 9114 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:51:26.750540 9105 server_base.cc:1034] running on GCE node
I20260806 22:51:26.750727 9105 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:51:26.750965 9105 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:51:26.752130 9105 hybrid_clock.cc:648] HybridClock initialized: now 1786056686752110 us; error 34 us; skew 500 ppm
Aug 06 22:51:26 dist-test-slave-1xrh krb5kdc[8489](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056686, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:51:26.755451 9105 init.cc:377] Logged in from keytab as kudu/127.28.218.193@KRBTEST.COM (short username kudu)
I20260806 22:51:26.756101 9105 webserver.cc:460] Webserver started at http://127.28.218.193:41785/ using document root <none> and password file <none>
I20260806 22:51:26.756402 9105 fs_manager.cc:359] Metadata directory not provided
I20260806 22:51:26.756563 9105 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:51:26.756747 9105 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 22:51:26 dist-test-slave-1xrh krb5kdc[8489](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056686, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.193@KRBTEST.COM for HTTP/127.28.218.212@KRBTEST.COM
I20260806 22:51:26.774243 9105 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/minicluster-data/ts-0/data/instance:
uuid: "46b2fdfe1df34700945220237aa0dfee"
format_stamp: "Formatted at 2026-08-06 22:51:26 on dist-test-slave-1xrh"
server_key: "7cccb89d77b92ae881de81aaaa8787e0"
server_key_iv: "69caa8953578fb35e3afac8f4664a86c"
server_key_version: "kuduclusterkey@0"
I20260806 22:51:26.774852 9105 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/minicluster-data/ts-0/wal/instance:
uuid: "46b2fdfe1df34700945220237aa0dfee"
format_stamp: "Formatted at 2026-08-06 22:51:26 on dist-test-slave-1xrh"
server_key: "7cccb89d77b92ae881de81aaaa8787e0"
server_key_iv: "69caa8953578fb35e3afac8f4664a86c"
server_key_version: "kuduclusterkey@0"
I20260806 22:51:26.777215 9105 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20260806 22:51:26.791181 9122 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:51:26.791468 9105 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20260806 22:51:26.791561 9105 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/minicluster-data/ts-0/wal
uuid: "46b2fdfe1df34700945220237aa0dfee"
format_stamp: "Formatted at 2026-08-06 22:51:26 on dist-test-slave-1xrh"
server_key: "7cccb89d77b92ae881de81aaaa8787e0"
server_key_iv: "69caa8953578fb35e3afac8f4664a86c"
server_key_version: "kuduclusterkey@0"
I20260806 22:51:26.791669 9105 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:51:26.800758 9105 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:51:26.802958 9105 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:51:26.803215 9105 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:51:26.803560 9105 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:51:26.804031 9105 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:51:26.804092 9105 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:51:26.804160 9105 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:51:26.804204 9105 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:51:26.817539 9105 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.193:33091
I20260806 22:51:26.817662 9241 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.193:33091 every 8 connection(s)
I20260806 22:51:26.818001 9105 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/minicluster-data/ts-0/data/info.pb
I20260806 22:51:26.823110 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 9105
I20260806 22:51:26.823269 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/minicluster-data/ts-0/wal/instance
I20260806 22:51:26.823493 29547 external_mini_cluster.cc:1428] Setting key 56e692b75d9300c2abf4ab8080adadca
Aug 06 22:51:26 dist-test-slave-1xrh krb5kdc[8489](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056686, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.193@KRBTEST.COM for kudu/127.28.218.254@KRBTEST.COM
I20260806 22:51:26.834273 9246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:26.818744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:36251 (local address 127.28.218.193:40871)
0806 22:51:26.820192 (+ 1448us) negotiation.cc:107] Waiting for socket to connect
0806 22:51:26.820239 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 22:51:26.820926 (+ 687us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:51:26.821468 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:51:26.821473 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:51:26.821855 (+ 382us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:51:26.822257 (+ 402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:51:26.822263 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:51:26.823504 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:51:26.823505 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:51:26.825217 (+ 1712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:51:26.825220 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:51:26.825731 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:51:26.825732 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:51:26.825962 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:26.826635 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:51:26.826645 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:51:26.828822 (+ 2177us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:51:26.832637 (+ 3815us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:51:26.832644 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:51:26.832647 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:51:26.832963 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:51:26.833195 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:51:26.833196 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:51:26.833196 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:51:26.833320 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:51:26.833615 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:51:26.833620 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:51:26.833784 (+ 164us) client_negotiation.cc:769] Sending connection context
0806 22:51:26.833887 (+ 103us) client_negotiation.cc:240] Negotiation successful
0806 22:51:26.834038 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1404,"thread_start_us":89,"threads_started":1}
I20260806 22:51:26.834784 9245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:26.818759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.193:40871 (local address 127.28.218.254:36251)
0806 22:51:26.821173 (+ 2414us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:26.821175 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:26.821183 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:51:26.821223 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:26.821224 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:26.821238 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:26.822359 (+ 1121us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:51:26.822425 (+ 66us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:26.823359 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:26.825325 (+ 1966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:26.825632 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:26.828933 (+ 3301us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:26.828993 (+ 60us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:51:26.828993 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:51:26.828994 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:51:26.828998 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:51:26.832531 (+ 3533us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:51:26.833058 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:51:26.833059 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:51:26.833059 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:51:26.833125 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:51:26.833406 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:51:26.833407 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:51:26.833408 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:51:26.833542 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:51:26.834553 (+ 1011us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:26.834596 (+ 43us) server_negotiation.cc:299] Negotiation successful
0806 22:51:26.834681 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2365,"thread_start_us":83,"threads_started":1}
I20260806 22:51:26.836098 9243 heartbeater.cc:347] Connected to a master server at 127.28.218.254:36251
I20260806 22:51:26.836193 9243 heartbeater.cc:464] Registering TS with master...
I20260806 22:51:26.836380 9243 heartbeater.cc:511] Master 127.28.218.254:36251 requested a full tablet report, sending...
I20260806 22:51:26.836941 9015 ts_manager.cc:194] Registered new tserver with Master: 46b2fdfe1df34700945220237aa0dfee (127.28.218.193:33091)
I20260806 22:51:26.837901 9015 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.218.193@KRBTEST.COM'} at 127.28.218.193:40871
WARNING: no policy specified for kudu/127.28.218.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.218.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.218.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.194.keytab.
Entry for principal kudu/127.28.218.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.194.keytab.
I20260806 22:51:26.862118 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.28.218.212:52807/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.218.194:0
--local_ip_for_outbound_sockets=127.28.218.194
--webserver_interface=127.28.218.194
--webserver_port=0
--tserver_master_addrs=127.28.218.254:36251
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.194.keytab
--principal=kudu/127.28.218.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.218.212:46329
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:51:26.997697 9250 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:51:26.997941 9250 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:51:26.997980 9250 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:51:26.999342 9250 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:51:26.999399 9250 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:51:26.999441 9250 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.218.194
I20260806 22:51:27.001022 9250 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:46329
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.28.218.212:52807/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.194.keytab
--principal=kudu/127.28.218.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.218.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.218.194
--webserver_port=0
--tserver_master_addrs=127.28.218.254:36251
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9250
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.218.194
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:51:27.001294 9250 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:51:27.001526 9250 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:51:27.007802 9258 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:51:27.007761 9257 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:51:27.007939 9261 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:51:27.008301 9250 server_base.cc:1034] running on GCE node
I20260806 22:51:27.008549 9250 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:51:27.008792 9250 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:51:27.009943 9250 hybrid_clock.cc:648] HybridClock initialized: now 1786056687009922 us; error 36 us; skew 500 ppm
Aug 06 22:51:27 dist-test-slave-1xrh krb5kdc[8489](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056687, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:51:27.013090 9250 init.cc:377] Logged in from keytab as kudu/127.28.218.194@KRBTEST.COM (short username kudu)
I20260806 22:51:27.013520 9250 webserver.cc:460] Webserver started at http://127.28.218.194:43625/ using document root <none> and password file <none>
I20260806 22:51:27.013746 9250 fs_manager.cc:359] Metadata directory not provided
I20260806 22:51:27.013814 9250 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:51:27.013952 9250 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 22:51:27 dist-test-slave-1xrh krb5kdc[8489](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056687, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.194@KRBTEST.COM for HTTP/127.28.218.212@KRBTEST.COM
I20260806 22:51:27.025985 9250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/minicluster-data/ts-1/data/instance:
uuid: "5b67fe015f7541948e2ef97f1c5f2f84"
format_stamp: "Formatted at 2026-08-06 22:51:27 on dist-test-slave-1xrh"
server_key: "92a24a79fb23fa0314bce07ec124a0e3"
server_key_iv: "65699668c5ad2958b5b5156258bb76ed"
server_key_version: "kuduclusterkey@0"
I20260806 22:51:27.026507 9250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/minicluster-data/ts-1/wal/instance:
uuid: "5b67fe015f7541948e2ef97f1c5f2f84"
format_stamp: "Formatted at 2026-08-06 22:51:27 on dist-test-slave-1xrh"
server_key: "92a24a79fb23fa0314bce07ec124a0e3"
server_key_iv: "65699668c5ad2958b5b5156258bb76ed"
server_key_version: "kuduclusterkey@0"
I20260806 22:51:27.032189 9250 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.003s
I20260806 22:51:27.048698 9268 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:51:27.048887 9250 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 22:51:27.048978 9250 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/minicluster-data/ts-1/wal
uuid: "5b67fe015f7541948e2ef97f1c5f2f84"
format_stamp: "Formatted at 2026-08-06 22:51:27 on dist-test-slave-1xrh"
server_key: "92a24a79fb23fa0314bce07ec124a0e3"
server_key_iv: "65699668c5ad2958b5b5156258bb76ed"
server_key_version: "kuduclusterkey@0"
I20260806 22:51:27.049078 9250 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
Aug 06 22:51:27 dist-test-slave-1xrh krb5kdc[8489](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786056687, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:51:27.065428 9250 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:51:27.068004 9250 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:51:27.068184 9250 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:51:27.068388 9250 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:51:27.068733 9250 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:51:27.068783 9250 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:51:27.068830 9250 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:51:27.068852 9250 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:51:27.083173 9250 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.194:41015
I20260806 22:51:27.083657 9250 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/minicluster-data/ts-1/data/info.pb
I20260806 22:51:27.084120 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 9250
I20260806 22:51:27.084242 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/minicluster-data/ts-1/wal/instance
I20260806 22:51:27.084455 29547 external_mini_cluster.cc:1428] Setting key b8886053d109d0293e96ca54eb0e8ac9
I20260806 22:51:27.084497 9386 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.194:41015 every 8 connection(s)
Aug 06 22:51:27 dist-test-slave-1xrh krb5kdc[8489](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056687, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.194@KRBTEST.COM for kudu/127.28.218.254@KRBTEST.COM
I20260806 22:51:27.099645 9245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:27.086156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.194:55297 (local address 127.28.218.254:36251)
0806 22:51:27.086220 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:27.086223 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:27.086665 (+ 442us) server_negotiation.cc:407] Connection header received
0806 22:51:27.087643 (+ 978us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:27.087644 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:27.087659 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:27.087690 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:51:27.088821 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:27.089609 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:27.091303 (+ 1694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:27.091617 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:27.091660 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:27.094231 (+ 2571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:51:27.094233 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:51:27.094233 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:51:27.094240 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:51:27.096255 (+ 2015us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:51:27.098529 (+ 2274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:51:27.098530 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:51:27.098530 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:51:27.098592 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:51:27.098881 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:51:27.098882 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:51:27.098882 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:51:27.099013 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:51:27.099053 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:27.099436 (+ 383us) server_negotiation.cc:299] Negotiation successful
0806 22:51:27.099525 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260806 22:51:27.100215 9389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:27.085841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:36251 (local address 127.28.218.194:55297)
0806 22:51:27.086535 (+ 694us) negotiation.cc:107] Waiting for socket to connect
0806 22:51:27.086581 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 22:51:27.087503 (+ 922us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:51:27.087880 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:51:27.087884 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:51:27.088287 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:51:27.088694 (+ 407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:51:27.088701 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:51:27.089726 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:51:27.089727 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:51:27.091170 (+ 1443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:51:27.091173 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:51:27.091749 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:51:27.091750 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:51:27.091902 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:27.092503 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:51:27.092513 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:51:27.094126 (+ 1613us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:51:27.096363 (+ 2237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:51:27.096368 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:51:27.096370 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:51:27.096642 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:51:27.098666 (+ 2024us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:51:27.098666 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:51:27.098667 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:51:27.098792 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:51:27.099097 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:51:27.099103 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:51:27.099273 (+ 170us) client_negotiation.cc:769] Sending connection context
0806 22:51:27.099779 (+ 506us) client_negotiation.cc:240] Negotiation successful
0806 22:51:27.099951 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":654,"thread_start_us":131,"threads_started":1}
I20260806 22:51:27.100654 9387 heartbeater.cc:347] Connected to a master server at 127.28.218.254:36251
I20260806 22:51:27.100749 9387 heartbeater.cc:464] Registering TS with master...
I20260806 22:51:27.100940 9387 heartbeater.cc:511] Master 127.28.218.254:36251 requested a full tablet report, sending...
I20260806 22:51:27.101284 9015 ts_manager.cc:194] Registered new tserver with Master: 5b67fe015f7541948e2ef97f1c5f2f84 (127.28.218.194:41015)
I20260806 22:51:27.101778 9015 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.218.194@KRBTEST.COM'} at 127.28.218.194:55297
WARNING: no policy specified for kudu/127.28.218.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.218.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.218.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.195.keytab.
Entry for principal kudu/127.28.218.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.195.keytab.
I20260806 22:51:27.134020 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.28.218.212:52807/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.218.195:0
--local_ip_for_outbound_sockets=127.28.218.195
--webserver_interface=127.28.218.195
--webserver_port=0
--tserver_master_addrs=127.28.218.254:36251
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.195.keytab
--principal=kudu/127.28.218.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.218.212:46329
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:51:27.250799 9392 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:51:27.251078 9392 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:51:27.251113 9392 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:51:27.252386 9392 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:51:27.252457 9392 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:51:27.252506 9392 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.218.195
I20260806 22:51:27.254597 9392 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:46329
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.28.218.212:52807/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.195.keytab
--principal=kudu/127.28.218.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.218.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.218.195
--webserver_port=0
--tserver_master_addrs=127.28.218.254:36251
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9392
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.218.195
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:51:27.254830 9392 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:51:27.255103 9392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:51:27.262048 9401 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:51:27.262048 9403 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:51:27.262336 9400 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:51:27.263010 9392 server_base.cc:1034] running on GCE node
I20260806 22:51:27.263243 9392 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:51:27.263523 9392 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:51:27.264684 9392 hybrid_clock.cc:648] HybridClock initialized: now 1786056687264672 us; error 25 us; skew 500 ppm
Aug 06 22:51:27 dist-test-slave-1xrh krb5kdc[8489](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056687, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:51:27.267896 9392 init.cc:377] Logged in from keytab as kudu/127.28.218.195@KRBTEST.COM (short username kudu)
I20260806 22:51:27.268456 9392 webserver.cc:460] Webserver started at http://127.28.218.195:34787/ using document root <none> and password file <none>
I20260806 22:51:27.268697 9392 fs_manager.cc:359] Metadata directory not provided
I20260806 22:51:27.268764 9392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:51:27.268899 9392 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 22:51:27 dist-test-slave-1xrh krb5kdc[8489](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056687, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.195@KRBTEST.COM for HTTP/127.28.218.212@KRBTEST.COM
I20260806 22:51:27.282579 9392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/minicluster-data/ts-2/data/instance:
uuid: "71a235890c7048438453b6e1948094a5"
format_stamp: "Formatted at 2026-08-06 22:51:27 on dist-test-slave-1xrh"
server_key: "58cb7c59f1ca91557afd426938f2d3da"
server_key_iv: "60e03e010198495e83cb49bc7f528d58"
server_key_version: "kuduclusterkey@0"
I20260806 22:51:27.283025 9392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/minicluster-data/ts-2/wal/instance:
uuid: "71a235890c7048438453b6e1948094a5"
format_stamp: "Formatted at 2026-08-06 22:51:27 on dist-test-slave-1xrh"
server_key: "58cb7c59f1ca91557afd426938f2d3da"
server_key_iv: "60e03e010198495e83cb49bc7f528d58"
server_key_version: "kuduclusterkey@0"
I20260806 22:51:27.290095 9392 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.000s sys 0.002s
I20260806 22:51:27.306910 9410 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:51:27.307082 9392 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.001s
I20260806 22:51:27.307197 9392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/minicluster-data/ts-2/wal
uuid: "71a235890c7048438453b6e1948094a5"
format_stamp: "Formatted at 2026-08-06 22:51:27 on dist-test-slave-1xrh"
server_key: "58cb7c59f1ca91557afd426938f2d3da"
server_key_iv: "60e03e010198495e83cb49bc7f528d58"
server_key_version: "kuduclusterkey@0"
I20260806 22:51:27.307838 9392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:51:27.324543 9392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:51:27.326943 9392 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:51:27.327081 9392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:51:27.327361 9392 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:51:27.327714 9392 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:51:27.327762 9392 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:51:27.327792 9392 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:51:27.327816 9392 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:51:27.339756 9392 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.195:41127
I20260806 22:51:27.340164 9392 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/minicluster-data/ts-2/data/info.pb
I20260806 22:51:27.344604 9531 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.195:41127 every 8 connection(s)
I20260806 22:51:27.349797 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 9392
I20260806 22:51:27.349936 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/minicluster-data/ts-2/wal/instance
I20260806 22:51:27.350215 29547 external_mini_cluster.cc:1428] Setting key 72e15673dbe0bb7f50d7684312d8f9f0
Aug 06 22:51:27 dist-test-slave-1xrh krb5kdc[8489](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056687, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.195@KRBTEST.COM for kudu/127.28.218.254@KRBTEST.COM
I20260806 22:51:27.357591 9245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:27.345382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.195:42119 (local address 127.28.218.254:36251)
0806 22:51:27.345427 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:27.345429 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:27.345747 (+ 318us) server_negotiation.cc:407] Connection header received
0806 22:51:27.346709 (+ 962us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:27.346710 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:27.346724 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:27.346756 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:51:27.347626 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:27.348484 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:27.350007 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:27.350218 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:27.350302 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:27.353646 (+ 3344us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:51:27.353647 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:51:27.353648 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:51:27.353656 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:51:27.355682 (+ 2026us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:51:27.356337 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:51:27.356338 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:51:27.356338 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:51:27.356411 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:51:27.356716 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:51:27.356718 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:51:27.356718 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:51:27.356895 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:51:27.356942 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:27.357315 (+ 373us) server_negotiation.cc:299] Negotiation successful
0806 22:51:27.357493 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 22:51:27.358417 9534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:27.345389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:36251 (local address 127.28.218.195:42119)
0806 22:51:27.345591 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0806 22:51:27.345633 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 22:51:27.346546 (+ 913us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:51:27.346958 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:51:27.346961 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:51:27.347222 (+ 261us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:51:27.347517 (+ 295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:51:27.347523 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:51:27.348602 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:51:27.348603 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:51:27.349842 (+ 1239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:51:27.349855 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:51:27.350308 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:51:27.350310 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:51:27.350453 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:27.351373 (+ 920us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:51:27.351383 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:51:27.353519 (+ 2136us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:51:27.355777 (+ 2258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:51:27.355798 (+ 21us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:51:27.355800 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:51:27.356233 (+ 433us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:51:27.356503 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:51:27.356504 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:51:27.356505 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:51:27.356640 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:51:27.356992 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:51:27.356998 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:51:27.357158 (+ 160us) client_negotiation.cc:769] Sending connection context
0806 22:51:27.357959 (+ 801us) client_negotiation.cc:240] Negotiation successful
0806 22:51:27.358126 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":80,"threads_started":1}
I20260806 22:51:27.362165 9532 heartbeater.cc:347] Connected to a master server at 127.28.218.254:36251
I20260806 22:51:27.362260 9532 heartbeater.cc:464] Registering TS with master...
I20260806 22:51:27.362439 9532 heartbeater.cc:511] Master 127.28.218.254:36251 requested a full tablet report, sending...
I20260806 22:51:27.362982 9015 ts_manager.cc:194] Registered new tserver with Master: 71a235890c7048438453b6e1948094a5 (127.28.218.195:41127)
I20260806 22:51:27.363500 9015 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.218.195@KRBTEST.COM'} at 127.28.218.195:42119
I20260806 22:51:27.373582 29547 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:51:27.384550 9245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:27.375421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60254 (local address 127.28.218.254:36251)
0806 22:51:27.375478 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:27.375481 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:27.375881 (+ 400us) server_negotiation.cc:407] Connection header received
0806 22:51:27.376023 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:27.376024 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:27.376036 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:27.376079 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:51:27.377102 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:27.377942 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:27.379110 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:27.379450 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:27.379516 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:27.381252 (+ 1736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:51:27.381253 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:51:27.381253 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:51:27.381261 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:51:27.383205 (+ 1944us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:51:27.383647 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:51:27.383648 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:51:27.383649 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:51:27.383710 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:51:27.383973 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:51:27.383974 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:51:27.383975 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:51:27.384123 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:51:27.384225 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:27.384377 (+ 152us) server_negotiation.cc:299] Negotiation successful
0806 22:51:27.384455 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 22:51:27.386399 9015 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60254:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:51:27.388902 9015 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:51:27.397512 9546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:27.391629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.195:41127 (local address 127.0.0.1:37462)
0806 22:51:27.392092 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0806 22:51:27.392131 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 22:51:27.392197 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:51:27.392673 (+ 476us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:51:27.392676 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:51:27.392678 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:51:27.392743 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:51:27.392744 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:51:27.394152 (+ 1408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:51:27.394153 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:51:27.395486 (+ 1333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:51:27.395489 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:51:27.397155 (+ 1666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:51:27.397156 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:51:27.397301 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:27.397316 (+ 15us) client_negotiation.cc:769] Sending connection context
0806 22:51:27.397389 (+ 73us) client_negotiation.cc:240] Negotiation successful
0806 22:51:27.397438 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":434,"thread_start_us":70,"threads_started":1}
I20260806 22:51:27.398085 9550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:27.392052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37462 (local address 127.28.218.195:41127)
0806 22:51:27.392448 (+ 396us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:27.392452 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:27.392463 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:51:27.392503 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:27.392505 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:27.392521 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:27.392593 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:51:27.392816 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:27.394054 (+ 1238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:27.395867 (+ 1813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:27.396878 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:27.397059 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:27.397118 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:27.397894 (+ 776us) server_negotiation.cc:299] Negotiation successful
0806 22:51:27.397938 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":360,"thread_start_us":77,"threads_started":1}
I20260806 22:51:27.398113 9547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:27.391663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.194:41015 (local address 127.0.0.1:43790)
0806 22:51:27.392267 (+ 604us) negotiation.cc:107] Waiting for socket to connect
0806 22:51:27.392278 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:51:27.392330 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:51:27.392879 (+ 549us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:51:27.392880 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:51:27.392893 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:51:27.392942 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:51:27.392943 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:51:27.395017 (+ 2074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:51:27.395018 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:51:27.396379 (+ 1361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:51:27.396383 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:51:27.397890 (+ 1507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:51:27.397891 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:51:27.397949 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:27.397953 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:51:27.398007 (+ 54us) client_negotiation.cc:240] Negotiation successful
0806 22:51:27.398046 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":552,"thread_start_us":89,"threads_started":1}
I20260806 22:51:27.398403 9548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:27.391905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43790 (local address 127.28.218.194:41015)
0806 22:51:27.392629 (+ 724us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:27.392633 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:27.392644 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:51:27.392683 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:27.392685 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:27.392700 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:27.392775 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:51:27.393049 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:27.394904 (+ 1855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:27.396545 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:27.397704 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:27.397799 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:27.397851 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:27.398286 (+ 435us) server_negotiation.cc:299] Negotiation successful
0806 22:51:27.398326 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":678,"thread_start_us":88,"threads_started":1}
I20260806 22:51:27.399025 9461 tablet_service.cc:1467] Processing CreateTablet for tablet ebac2944f67e4d94be4afae20a5a6116 (DEFAULT_TABLE table=test-table [id=a6b1d5e09766464e9ee7ca90734f2388]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:51:27.399395 9461 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ebac2944f67e4d94be4afae20a5a6116. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:51:27.399628 9317 tablet_service.cc:1467] Processing CreateTablet for tablet ebac2944f67e4d94be4afae20a5a6116 (DEFAULT_TABLE table=test-table [id=a6b1d5e09766464e9ee7ca90734f2388]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:51:27.399942 9317 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ebac2944f67e4d94be4afae20a5a6116. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:51:27.404748 9552 tablet_bootstrap.cc:492] T ebac2944f67e4d94be4afae20a5a6116 P 5b67fe015f7541948e2ef97f1c5f2f84: Bootstrap starting.
I20260806 22:51:27.407069 9553 tablet_bootstrap.cc:492] T ebac2944f67e4d94be4afae20a5a6116 P 71a235890c7048438453b6e1948094a5: Bootstrap starting.
I20260806 22:51:27.407774 9552 tablet_bootstrap.cc:654] T ebac2944f67e4d94be4afae20a5a6116 P 5b67fe015f7541948e2ef97f1c5f2f84: Neither blocks nor log segments found. Creating new log.
I20260806 22:51:27.408938 9545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:27.391821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.193:33091 (local address 127.0.0.1:35906)
0806 22:51:27.395605 (+ 3784us) negotiation.cc:107] Waiting for socket to connect
0806 22:51:27.395614 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:51:27.395652 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:51:27.404856 (+ 9204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:51:27.404857 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:51:27.404859 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:51:27.404914 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:51:27.404916 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:51:27.406393 (+ 1477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:51:27.406394 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:51:27.407604 (+ 1210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:51:27.407607 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:51:27.408677 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:51:27.408679 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:51:27.408739 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:27.408743 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:51:27.408808 (+ 65us) client_negotiation.cc:240] Negotiation successful
0806 22:51:27.408844 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3768,"thread_start_us":87,"threads_started":1}
I20260806 22:51:27.409536 9551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:27.393126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35906 (local address 127.28.218.193:33091)
0806 22:51:27.404623 (+ 11497us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:27.404629 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:27.404641 (+ 12us) server_negotiation.cc:407] Connection header received
0806 22:51:27.404678 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:27.404690 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:27.404708 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:27.405080 (+ 372us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:51:27.405174 (+ 94us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:27.406271 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:27.407763 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:27.408537 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:27.409168 (+ 631us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:27.409363 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:27.409391 (+ 28us) server_negotiation.cc:299] Negotiation successful
0806 22:51:27.409450 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11452,"thread_start_us":93,"threads_started":1}
I20260806 22:51:27.409724 9552 log.cc:824] T ebac2944f67e4d94be4afae20a5a6116 P 5b67fe015f7541948e2ef97f1c5f2f84: Log is configured to *not* fsync() on all Append() calls
I20260806 22:51:27.410328 9174 tablet_service.cc:1467] Processing CreateTablet for tablet ebac2944f67e4d94be4afae20a5a6116 (DEFAULT_TABLE table=test-table [id=a6b1d5e09766464e9ee7ca90734f2388]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:51:27.410750 9174 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ebac2944f67e4d94be4afae20a5a6116. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:51:27.411471 9553 tablet_bootstrap.cc:654] T ebac2944f67e4d94be4afae20a5a6116 P 71a235890c7048438453b6e1948094a5: Neither blocks nor log segments found. Creating new log.
I20260806 22:51:27.412117 9553 log.cc:824] T ebac2944f67e4d94be4afae20a5a6116 P 71a235890c7048438453b6e1948094a5: Log is configured to *not* fsync() on all Append() calls
I20260806 22:51:27.414461 9557 tablet_bootstrap.cc:492] T ebac2944f67e4d94be4afae20a5a6116 P 46b2fdfe1df34700945220237aa0dfee: Bootstrap starting.
I20260806 22:51:27.415441 9552 tablet_bootstrap.cc:492] T ebac2944f67e4d94be4afae20a5a6116 P 5b67fe015f7541948e2ef97f1c5f2f84: No bootstrap required, opened a new log
I20260806 22:51:27.415524 9552 ts_tablet_manager.cc:1397] T ebac2944f67e4d94be4afae20a5a6116 P 5b67fe015f7541948e2ef97f1c5f2f84: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.000s
I20260806 22:51:27.415547 9557 tablet_bootstrap.cc:654] T ebac2944f67e4d94be4afae20a5a6116 P 46b2fdfe1df34700945220237aa0dfee: Neither blocks nor log segments found. Creating new log.
I20260806 22:51:27.415942 9553 tablet_bootstrap.cc:492] T ebac2944f67e4d94be4afae20a5a6116 P 71a235890c7048438453b6e1948094a5: No bootstrap required, opened a new log
I20260806 22:51:27.416018 9553 ts_tablet_manager.cc:1397] T ebac2944f67e4d94be4afae20a5a6116 P 71a235890c7048438453b6e1948094a5: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.003s
I20260806 22:51:27.416055 9557 log.cc:824] T ebac2944f67e4d94be4afae20a5a6116 P 46b2fdfe1df34700945220237aa0dfee: Log is configured to *not* fsync() on all Append() calls
I20260806 22:51:27.418288 9553 raft_consensus.cc:348] T ebac2944f67e4d94be4afae20a5a6116 P 71a235890c7048438453b6e1948094a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b67fe015f7541948e2ef97f1c5f2f84" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 41015 } } peers { permanent_uuid: "71a235890c7048438453b6e1948094a5" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 41127 } } peers { permanent_uuid: "46b2fdfe1df34700945220237aa0dfee" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 33091 } }
I20260806 22:51:27.418406 9553 raft_consensus.cc:374] T ebac2944f67e4d94be4afae20a5a6116 P 71a235890c7048438453b6e1948094a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:51:27.418426 9553 raft_consensus.cc:729] T ebac2944f67e4d94be4afae20a5a6116 P 71a235890c7048438453b6e1948094a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71a235890c7048438453b6e1948094a5, State: Initialized, Role: FOLLOWER
I20260806 22:51:27.418560 9553 consensus_queue.cc:260] T ebac2944f67e4d94be4afae20a5a6116 P 71a235890c7048438453b6e1948094a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b67fe015f7541948e2ef97f1c5f2f84" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 41015 } } peers { permanent_uuid: "71a235890c7048438453b6e1948094a5" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 41127 } } peers { permanent_uuid: "46b2fdfe1df34700945220237aa0dfee" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 33091 } }
I20260806 22:51:27.418877 9553 ts_tablet_manager.cc:1428] T ebac2944f67e4d94be4afae20a5a6116 P 71a235890c7048438453b6e1948094a5: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260806 22:51:27.419004 9532 heartbeater.cc:503] Master 127.28.218.254:36251 was elected leader, sending a full tablet report...
I20260806 22:51:27.418923 9552 raft_consensus.cc:348] T ebac2944f67e4d94be4afae20a5a6116 P 5b67fe015f7541948e2ef97f1c5f2f84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b67fe015f7541948e2ef97f1c5f2f84" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 41015 } } peers { permanent_uuid: "71a235890c7048438453b6e1948094a5" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 41127 } } peers { permanent_uuid: "46b2fdfe1df34700945220237aa0dfee" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 33091 } }
I20260806 22:51:27.419085 9552 raft_consensus.cc:374] T ebac2944f67e4d94be4afae20a5a6116 P 5b67fe015f7541948e2ef97f1c5f2f84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:51:27.419117 9552 raft_consensus.cc:729] T ebac2944f67e4d94be4afae20a5a6116 P 5b67fe015f7541948e2ef97f1c5f2f84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b67fe015f7541948e2ef97f1c5f2f84, State: Initialized, Role: FOLLOWER
I20260806 22:51:27.419185 9557 tablet_bootstrap.cc:492] T ebac2944f67e4d94be4afae20a5a6116 P 46b2fdfe1df34700945220237aa0dfee: No bootstrap required, opened a new log
I20260806 22:51:27.419247 9557 ts_tablet_manager.cc:1397] T ebac2944f67e4d94be4afae20a5a6116 P 46b2fdfe1df34700945220237aa0dfee: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260806 22:51:27.419265 9552 consensus_queue.cc:260] T ebac2944f67e4d94be4afae20a5a6116 P 5b67fe015f7541948e2ef97f1c5f2f84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b67fe015f7541948e2ef97f1c5f2f84" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 41015 } } peers { permanent_uuid: "71a235890c7048438453b6e1948094a5" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 41127 } } peers { permanent_uuid: "46b2fdfe1df34700945220237aa0dfee" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 33091 } }
I20260806 22:51:27.419575 9552 ts_tablet_manager.cc:1428] T ebac2944f67e4d94be4afae20a5a6116 P 5b67fe015f7541948e2ef97f1c5f2f84: Time spent starting tablet: real 0.004s user 0.000s sys 0.004s
I20260806 22:51:27.419935 9387 heartbeater.cc:503] Master 127.28.218.254:36251 was elected leader, sending a full tablet report...
I20260806 22:51:27.422650 9557 raft_consensus.cc:348] T ebac2944f67e4d94be4afae20a5a6116 P 46b2fdfe1df34700945220237aa0dfee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b67fe015f7541948e2ef97f1c5f2f84" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 41015 } } peers { permanent_uuid: "71a235890c7048438453b6e1948094a5" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 41127 } } peers { permanent_uuid: "46b2fdfe1df34700945220237aa0dfee" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 33091 } }
I20260806 22:51:27.422757 9557 raft_consensus.cc:374] T ebac2944f67e4d94be4afae20a5a6116 P 46b2fdfe1df34700945220237aa0dfee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:51:27.422777 9557 raft_consensus.cc:729] T ebac2944f67e4d94be4afae20a5a6116 P 46b2fdfe1df34700945220237aa0dfee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46b2fdfe1df34700945220237aa0dfee, State: Initialized, Role: FOLLOWER
I20260806 22:51:27.422879 9557 consensus_queue.cc:260] T ebac2944f67e4d94be4afae20a5a6116 P 46b2fdfe1df34700945220237aa0dfee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b67fe015f7541948e2ef97f1c5f2f84" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 41015 } } peers { permanent_uuid: "71a235890c7048438453b6e1948094a5" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 41127 } } peers { permanent_uuid: "46b2fdfe1df34700945220237aa0dfee" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 33091 } }
I20260806 22:51:27.423192 9557 ts_tablet_manager.cc:1428] T ebac2944f67e4d94be4afae20a5a6116 P 46b2fdfe1df34700945220237aa0dfee: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260806 22:51:27.423254 9243 heartbeater.cc:503] Master 127.28.218.254:36251 was elected leader, sending a full tablet report...
I20260806 22:51:27.523923 9560 raft_consensus.cc:482] T ebac2944f67e4d94be4afae20a5a6116 P 5b67fe015f7541948e2ef97f1c5f2f84 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:51:27.524084 9560 raft_consensus.cc:504] T ebac2944f67e4d94be4afae20a5a6116 P 5b67fe015f7541948e2ef97f1c5f2f84 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b67fe015f7541948e2ef97f1c5f2f84" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 41015 } } peers { permanent_uuid: "71a235890c7048438453b6e1948094a5" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 41127 } } peers { permanent_uuid: "46b2fdfe1df34700945220237aa0dfee" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 33091 } }
I20260806 22:51:27.524502 9560 leader_election.cc:288] T ebac2944f67e4d94be4afae20a5a6116 P 5b67fe015f7541948e2ef97f1c5f2f84 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 71a235890c7048438453b6e1948094a5 (127.28.218.195:41127), 46b2fdfe1df34700945220237aa0dfee (127.28.218.193:33091)
I20260806 22:51:27.529076 9389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:27.524802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.195:41127 (local address 127.28.218.194:41081)
0806 22:51:27.525184 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0806 22:51:27.525199 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 22:51:27.525253 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:51:27.525568 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:51:27.525568 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:51:27.525570 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:51:27.525647 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:51:27.525650 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:51:27.526635 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:51:27.526636 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:51:27.527898 (+ 1262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:51:27.527901 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:51:27.528734 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:51:27.528735 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:51:27.528816 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:27.528820 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:51:27.528869 (+ 49us) client_negotiation.cc:240] Negotiation successful
0806 22:51:27.528949 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":355,"mutex_wait_us":6}
I20260806 22:51:27.529204 9551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:27.525233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.194:47303 (local address 127.28.218.193:33091)
0806 22:51:27.525330 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:27.525331 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:27.525456 (+ 125us) server_negotiation.cc:407] Connection header received
0806 22:51:27.525556 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:27.525557 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:27.525568 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:27.525596 (+ 28us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:51:27.525872 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:27.526559 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:27.528154 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:27.528747 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:27.528788 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:27.528795 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:27.529071 (+ 276us) server_negotiation.cc:299] Negotiation successful
0806 22:51:27.529124 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260806 22:51:27.529358 9550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:27.525349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.194:41081 (local address 127.28.218.195:41127)
0806 22:51:27.525397 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:27.525399 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:27.525424 (+ 25us) server_negotiation.cc:407] Connection header received
0806 22:51:27.525452 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:27.525453 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:27.525464 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:27.525515 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:51:27.525724 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:27.526520 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:27.527996 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:27.528619 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:27.528692 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:27.528701 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:27.529231 (+ 530us) server_negotiation.cc:299] Negotiation successful
0806 22:51:27.529278 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260806 22:51:27.529671 9484 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ebac2944f67e4d94be4afae20a5a6116" candidate_uuid: "5b67fe015f7541948e2ef97f1c5f2f84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71a235890c7048438453b6e1948094a5" is_pre_election: true
I20260806 22:51:27.529843 9484 raft_consensus.cc:2454] T ebac2944f67e4d94be4afae20a5a6116 P 71a235890c7048438453b6e1948094a5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b67fe015f7541948e2ef97f1c5f2f84 in term 0.
I20260806 22:51:27.530036 9562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:27.524921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.193:33091 (local address 127.28.218.194:47303)
0806 22:51:27.525399 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0806 22:51:27.525409 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:51:27.525481 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:51:27.525661 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:51:27.525661 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:51:27.525663 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:51:27.525784 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:51:27.525787 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:51:27.526648 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:51:27.526649 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:51:27.528039 (+ 1390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:51:27.528041 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:51:27.528850 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:51:27.528850 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:51:27.528979 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:27.528982 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:51:27.529733 (+ 751us) client_negotiation.cc:240] Negotiation successful
0806 22:51:27.529834 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":462,"mutex_wait_us":149,"thread_start_us":119,"threads_started":1}
I20260806 22:51:27.530086 9271 leader_election.cc:302] T ebac2944f67e4d94be4afae20a5a6116 P 5b67fe015f7541948e2ef97f1c5f2f84 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b67fe015f7541948e2ef97f1c5f2f84, 71a235890c7048438453b6e1948094a5; no voters:
I20260806 22:51:27.530249 9560 raft_consensus.cc:2781] T ebac2944f67e4d94be4afae20a5a6116 P 5b67fe015f7541948e2ef97f1c5f2f84 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:51:27.530300 9560 raft_consensus.cc:482] T ebac2944f67e4d94be4afae20a5a6116 P 5b67fe015f7541948e2ef97f1c5f2f84 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:51:27.530330 9560 raft_consensus.cc:3037] T ebac2944f67e4d94be4afae20a5a6116 P 5b67fe015f7541948e2ef97f1c5f2f84 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:51:27.530489 9194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ebac2944f67e4d94be4afae20a5a6116" candidate_uuid: "5b67fe015f7541948e2ef97f1c5f2f84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46b2fdfe1df34700945220237aa0dfee" is_pre_election: true
I20260806 22:51:27.530635 9194 raft_consensus.cc:2454] T ebac2944f67e4d94be4afae20a5a6116 P 46b2fdfe1df34700945220237aa0dfee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b67fe015f7541948e2ef97f1c5f2f84 in term 0.
I20260806 22:51:27.531538 9560 raft_consensus.cc:504] T ebac2944f67e4d94be4afae20a5a6116 P 5b67fe015f7541948e2ef97f1c5f2f84 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b67fe015f7541948e2ef97f1c5f2f84" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 41015 } } peers { permanent_uuid: "71a235890c7048438453b6e1948094a5" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 41127 } } peers { permanent_uuid: "46b2fdfe1df34700945220237aa0dfee" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 33091 } }
I20260806 22:51:27.531672 9560 leader_election.cc:288] T ebac2944f67e4d94be4afae20a5a6116 P 5b67fe015f7541948e2ef97f1c5f2f84 [CANDIDATE]: Term 1 election: Requested vote from peers 71a235890c7048438453b6e1948094a5 (127.28.218.195:41127), 46b2fdfe1df34700945220237aa0dfee (127.28.218.193:33091)
I20260806 22:51:27.531859 9484 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ebac2944f67e4d94be4afae20a5a6116" candidate_uuid: "5b67fe015f7541948e2ef97f1c5f2f84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71a235890c7048438453b6e1948094a5"
I20260806 22:51:27.531883 9194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ebac2944f67e4d94be4afae20a5a6116" candidate_uuid: "5b67fe015f7541948e2ef97f1c5f2f84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46b2fdfe1df34700945220237aa0dfee"
I20260806 22:51:27.531939 9484 raft_consensus.cc:3037] T ebac2944f67e4d94be4afae20a5a6116 P 71a235890c7048438453b6e1948094a5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:51:27.531973 9194 raft_consensus.cc:3037] T ebac2944f67e4d94be4afae20a5a6116 P 46b2fdfe1df34700945220237aa0dfee [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:51:27.533031 9194 raft_consensus.cc:2454] T ebac2944f67e4d94be4afae20a5a6116 P 46b2fdfe1df34700945220237aa0dfee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b67fe015f7541948e2ef97f1c5f2f84 in term 1.
I20260806 22:51:27.533217 9271 leader_election.cc:302] T ebac2944f67e4d94be4afae20a5a6116 P 5b67fe015f7541948e2ef97f1c5f2f84 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46b2fdfe1df34700945220237aa0dfee, 5b67fe015f7541948e2ef97f1c5f2f84; no voters:
I20260806 22:51:27.533257 9484 raft_consensus.cc:2454] T ebac2944f67e4d94be4afae20a5a6116 P 71a235890c7048438453b6e1948094a5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b67fe015f7541948e2ef97f1c5f2f84 in term 1.
I20260806 22:51:27.533298 9560 raft_consensus.cc:2781] T ebac2944f67e4d94be4afae20a5a6116 P 5b67fe015f7541948e2ef97f1c5f2f84 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:51:27.533605 9560 raft_consensus.cc:686] T ebac2944f67e4d94be4afae20a5a6116 P 5b67fe015f7541948e2ef97f1c5f2f84 [term 1 LEADER]: Becoming Leader. State: Replica: 5b67fe015f7541948e2ef97f1c5f2f84, State: Running, Role: LEADER
I20260806 22:51:27.533720 9560 consensus_queue.cc:237] T ebac2944f67e4d94be4afae20a5a6116 P 5b67fe015f7541948e2ef97f1c5f2f84 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b67fe015f7541948e2ef97f1c5f2f84" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 41015 } } peers { permanent_uuid: "71a235890c7048438453b6e1948094a5" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 41127 } } peers { permanent_uuid: "46b2fdfe1df34700945220237aa0dfee" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 33091 } }
I20260806 22:51:27.534582 9015 catalog_manager.cc:5495] T ebac2944f67e4d94be4afae20a5a6116 P 5b67fe015f7541948e2ef97f1c5f2f84 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5b67fe015f7541948e2ef97f1c5f2f84 (127.28.218.194). New cstate: current_term: 1 leader_uuid: "5b67fe015f7541948e2ef97f1c5f2f84" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b67fe015f7541948e2ef97f1c5f2f84" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 41015 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "71a235890c7048438453b6e1948094a5" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 41127 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46b2fdfe1df34700945220237aa0dfee" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 33091 } health_report { overall_health: UNKNOWN } } }
I20260806 22:51:27.549732 9548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:27.545967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43798 (local address 127.28.218.194:41015)
0806 22:51:27.546008 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0806 22:51:27.546010 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:51:27.546020 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:51:27.546083 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:51:27.546084 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:51:27.546104 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:51:27.546186 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:51:27.546357 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:27.547151 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:27.548225 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:51:27.548511 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:51:27.548642 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:51:27.548851 (+ 209us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:51:27.549038 (+ 187us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:51:27.549161 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0806 22:51:27.549555 (+ 394us) server_negotiation.cc:299] Negotiation successful
0806 22:51:27.549594 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 22:51:27.551993 9194 raft_consensus.cc:1261] T ebac2944f67e4d94be4afae20a5a6116 P 46b2fdfe1df34700945220237aa0dfee [term 1 FOLLOWER]: Refusing update from remote peer 5b67fe015f7541948e2ef97f1c5f2f84: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:51:27.552011 9484 raft_consensus.cc:1261] T ebac2944f67e4d94be4afae20a5a6116 P 71a235890c7048438453b6e1948094a5 [term 1 FOLLOWER]: Refusing update from remote peer 5b67fe015f7541948e2ef97f1c5f2f84: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:51:27.552393 9563 consensus_queue.cc:1035] T ebac2944f67e4d94be4afae20a5a6116 P 5b67fe015f7541948e2ef97f1c5f2f84 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46b2fdfe1df34700945220237aa0dfee" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 33091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:51:27.552502 9560 consensus_queue.cc:1035] T ebac2944f67e4d94be4afae20a5a6116 P 5b67fe015f7541948e2ef97f1c5f2f84 [LEADER]: Connected to new peer: Peer: permanent_uuid: "71a235890c7048438453b6e1948094a5" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 41127 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:51:27.559311 9566 mvcc.cc:205] Tried to move back new op lower bound from 7315688192209948672 to 7315688192138952704. Current Snapshot: MvccSnapshot[applied={T|T < 7315688192209948672}]
I20260806 22:51:27.562176 9015 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60254:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:51:27.562276 9015 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60254:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:51:27.563244 9015 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0eb6e1a9c254404888f746c6b82dca6f: Sending DeleteTablet for 3 replicas of tablet ebac2944f67e4d94be4afae20a5a6116
I20260806 22:51:27.563658 9317 tablet_service.cc:1514] Processing DeleteTablet for tablet ebac2944f67e4d94be4afae20a5a6116 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:51:27 UTC) from {username='kudu', principal='kudu/127.28.218.254@KRBTEST.COM'} at 127.0.0.1:43790
I20260806 22:51:27.564369 9461 tablet_service.cc:1514] Processing DeleteTablet for tablet ebac2944f67e4d94be4afae20a5a6116 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:51:27 UTC) from {username='kudu', principal='kudu/127.28.218.254@KRBTEST.COM'} at 127.0.0.1:37462
I20260806 22:51:27.564345 9174 tablet_service.cc:1514] Processing DeleteTablet for tablet ebac2944f67e4d94be4afae20a5a6116 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:51:27 UTC) from {username='kudu', principal='kudu/127.28.218.254@KRBTEST.COM'} at 127.0.0.1:35906
I20260806 22:51:27.564977 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 9105
I20260806 22:51:27.568269 9567 mvcc.cc:205] Tried to move back new op lower bound from 7315688192209948672 to 7315688192138952704. Current Snapshot: MvccSnapshot[applied={T|T < 7315688192209948672}]
I20260806 22:51:27.579326 9577 tablet_replica.cc:330] T ebac2944f67e4d94be4afae20a5a6116 P 5b67fe015f7541948e2ef97f1c5f2f84: stopping tablet replica
I20260806 22:51:27.579602 9578 tablet_replica.cc:330] T ebac2944f67e4d94be4afae20a5a6116 P 71a235890c7048438453b6e1948094a5: stopping tablet replica
I20260806 22:51:27.579748 9578 raft_consensus.cc:2229] T ebac2944f67e4d94be4afae20a5a6116 P 71a235890c7048438453b6e1948094a5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:51:27.579896 9577 raft_consensus.cc:2229] T ebac2944f67e4d94be4afae20a5a6116 P 5b67fe015f7541948e2ef97f1c5f2f84 [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:51:27.580057 9578 raft_consensus.cc:2258] T ebac2944f67e4d94be4afae20a5a6116 P 71a235890c7048438453b6e1948094a5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:51:27.580193 9577 raft_consensus.cc:2258] T ebac2944f67e4d94be4afae20a5a6116 P 5b67fe015f7541948e2ef97f1c5f2f84 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:51:27.580849 9577 ts_tablet_manager.cc:1905] T ebac2944f67e4d94be4afae20a5a6116 P 5b67fe015f7541948e2ef97f1c5f2f84: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:51:27.581142 9578 ts_tablet_manager.cc:1905] T ebac2944f67e4d94be4afae20a5a6116 P 71a235890c7048438453b6e1948094a5: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260806 22:51:27.581446 8999 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.28.218.193:33091 (error 108)
I20260806 22:51:27.582414 9545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:51:27.582285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.193:33091 (local address 127.0.0.1:35908)
0806 22:51:27.582330 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0806 22:51:27.582381 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.193:33091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260806 22:51:27.582434 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 9250
W20260806 22:51:27.582517 8999 catalog_manager.cc:4584] TS 46b2fdfe1df34700945220237aa0dfee (127.28.218.193:33091): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ebac2944f67e4d94be4afae20a5a6116: Network error: Client connection negotiation failed: client connection to 127.28.218.193:33091: connect: Connection refused (error 111)
I20260806 22:51:27.586915 9578 ts_tablet_manager.cc:1918] T ebac2944f67e4d94be4afae20a5a6116 P 71a235890c7048438453b6e1948094a5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:51:27.587018 9578 log.cc:1196] T ebac2944f67e4d94be4afae20a5a6116 P 71a235890c7048438453b6e1948094a5: Deleting WAL directory at /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056586788339-29547-0/minicluster-data/ts-2/wal/wals/ebac2944f67e4d94be4afae20a5a6116
I20260806 22:51:27.587716 9578 ts_tablet_manager.cc:1939] T ebac2944f67e4d94be4afae20a5a6116 P 71a235890c7048438453b6e1948094a5: Deleting consensus metadata
I20260806 22:51:27.588213 8999 catalog_manager.cc:4841] TS 71a235890c7048438453b6e1948094a5 (127.28.218.195:41127): tablet ebac2944f67e4d94be4afae20a5a6116 (table test-table [id=a6b1d5e09766464e9ee7ca90734f2388]) successfully deleted
W20260806 22:51:27.592895 9545 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:51:27.592745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.194:41015 (local address 127.0.0.1:43810)
0806 22:51:27.592809 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0806 22:51:27.592846 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.194:41015: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":41}
I20260806 22:51:27.593155 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 9392
I20260806 22:51:27.599457 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 8979
I20260806 22:51:27.608052 29547 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260806 22:51:27.952165 29547 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260806 22:51:27.952271 29547 mini_ranger.cc:66] Stopping Ranger...
I20260806 22:51:28.297783 29547 mini_ranger.cc:68] Stopped Ranger
2026-08-06 22:51:28.298 UTC [8520] LOG: received smart shutdown request
2026-08-06 22:51:28.302 UTC [8520] LOG: background worker "logical replication launcher" (PID 8529) exited with exit code 1
2026-08-06 22:51:28.302 UTC [8524] LOG: shutting down
2026-08-06 22:51:28.471 UTC [8520] LOG: database system is shut down
2026-08-06T22:51:28Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (46350 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:51:28 dist-test-slave-1xrh krb5kdc[9589](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:51:28 dist-test-slave-1xrh krb5kdc[9589](info): set up 2 sockets
Aug 06 22:51:28 dist-test-slave-1xrh krb5kdc[9589](info): commencing operation
krb5kdc: starting...
W20260806 22:51:29.658990 29547 mini_kdc.cc:121] Time spent starting KDC: real 1.051s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/krb5kdc/test-user.keytab.
Aug 06 22:51:29 dist-test-slave-1xrh krb5kdc[9589](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056689, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:51:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:51:29Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.28.218.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.28.218.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.28.218.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/krb5kdc/rangeradmin_127.28.218.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.28.218.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/krb5kdc/rangeradmin_127.28.218.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.28.218.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.28.218.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.28.218.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/krb5kdc/rangerlookup_127.28.218.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.28.218.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/krb5kdc/rangerlookup_127.28.218.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.28.218.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.28.218.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.28.218.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/krb5kdc/HTTP_127.28.218.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.28.218.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/krb5kdc/HTTP_127.28.218.212@KRBTEST.COM.keytab.
I20260806 22:51:29.823812 29547 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskQja9Us/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/postgres -l logfile start
2026-08-06 22:51:32.282 UTC [9620] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-06 22:51:32.282 UTC [9620] LOG: listening on IPv4 address "127.28.218.212", port 56393
2026-08-06 22:51:32.286 UTC [9620] LOG: listening on Unix socket "/tmp/.s.PGSQL.56393"
2026-08-06 22:51:32.321 UTC [9623] LOG: database system was shut down at 2026-08-06 22:51:30 UTC
2026-08-06 22:51:32.325 UTC [9620] LOG: database system is ready to accept connections
I20260806 22:51:33.290987 29547 mini_postgres.cc:96] Postgres bound to 56393
2026-08-06 22:51:33.297 UTC [9631] FATAL: database "slave" does not exist
127.28.218.212:56393 - accepting connections
I20260806 22:51:33.298115 29547 mini_ranger.cc:161] Starting Ranger...
I20260806 22:51:33.312932 29547 mini_ranger.cc:84] Created miniranger Postgres user
I20260806 22:51:33.773743 29547 mini_ranger.cc:87] Created ranger Postgres database
I20260806 22:51:33.773837 29547 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/ranger-admin
2026-08-06 22:51:34,058 [I] DB FLAVOR :POSTGRES
2026-08-06 22:51:34,059 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 22:51:34,059 [I] Checking connection..
2026-08-06 22:51:34,059 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQja9Us/build/release/bin/postgresql.jar:/tmp/dist-test-taskQja9Us/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.218.212:56393/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-06 22:51:34,310 [I] Checking connection passed.
2026-08-06 22:51:34,310 [I] --------- Verifying version history table ---------
2026-08-06 22:51:34,311 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQja9Us/build/release/bin/postgresql.jar:/tmp/dist-test-taskQja9Us/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.218.212:56393/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 22:51:34,564 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 22:51:34,565 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQja9Us/build/release/bin/postgresql.jar:/tmp/dist-test-taskQja9Us/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.218.212:56393/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 22:51:34,819 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 22:51:34,820 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-06 22:51:34,820 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQja9Us/build/release/bin/postgresql.jar:/tmp/dist-test-taskQja9Us/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.218.212:56393/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskQja9Us/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-06 22:51:35.071 UTC [9701] WARNING: there is no transaction in progress
2026-08-06 22:51:35,078 [I] create_dbversion_catalog.sql file imported successfully
2026-08-06 22:51:35,079 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQja9Us/build/release/bin/postgresql.jar:/tmp/dist-test-taskQja9Us/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.218.212:56393/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 22:51:35,337 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-06 22:51:35,337 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 22:51:35,337 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQja9Us/build/release/bin/postgresql.jar:/tmp/dist-test-taskQja9Us/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.218.212:56393/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-06 22:51:35,557 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQja9Us/build/release/bin/postgresql.jar:/tmp/dist-test-taskQja9Us/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.218.212:56393/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-06 22:51:35,806 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQja9Us/build/release/bin/postgresql.jar:/tmp/dist-test-taskQja9Us/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.218.212:56393/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-1xrh.c.gcp-upstream.internal','N') ;"
2026-08-06 22:51:36,039 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQja9Us/build/release/bin/postgresql.jar:/tmp/dist-test-taskQja9Us/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.218.212:56393/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 22:51:36,269 [I] Table x_portal_user does not exist in database ranger
2026-08-06 22:51:36,269 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQja9Us/build/release/bin/postgresql.jar:/tmp/dist-test-taskQja9Us/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.218.212:56393/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 22:51:36,518 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-06 22:51:36,518 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQja9Us/build/release/bin/postgresql.jar:/tmp/dist-test-taskQja9Us/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.218.212:56393/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 22:51:36,770 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQja9Us/build/release/bin/postgresql.jar:/tmp/dist-test-taskQja9Us/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.218.212:56393/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 22:51:37,018 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-06 22:51:37,018 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQja9Us/build/release/bin/postgresql.jar:/tmp/dist-test-taskQja9Us/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.218.212:56393/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskQja9Us/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-06 22:51:37.959 UTC [9814] WARNING: there is no transaction in progress
2026-08-06 22:51:37.976 UTC [9814] WARNING: there is no transaction in progress
2026-08-06 22:51:37.993 UTC [9814] WARNING: there is no transaction in progress
2026-08-06 22:51:38.008 UTC [9814] WARNING: there is no transaction in progress
2026-08-06 22:51:38.023 UTC [9814] WARNING: there is no transaction in progress
2026-08-06 22:51:38.040 UTC [9814] WARNING: there is no transaction in progress
2026-08-06 22:51:38.091 UTC [9814] WARNING: there is no transaction in progress
2026-08-06 22:51:38.097 UTC [9814] WARNING: there is no transaction in progress
2026-08-06 22:51:38.111 UTC [9814] WARNING: there is no transaction in progress
2026-08-06 22:51:38.121 UTC [9814] WARNING: there is no transaction in progress
2026-08-06 22:51:38.130 UTC [9814] WARNING: there is no transaction in progress
2026-08-06 22:51:38.144 UTC [9814] WARNING: there is no transaction in progress
2026-08-06 22:51:38.153 UTC [9814] WARNING: there is no transaction in progress
2026-08-06 22:51:38.159 UTC [9814] WARNING: there is no transaction in progress
2026-08-06 22:51:38.166 UTC [9814] WARNING: there is no transaction in progress
2026-08-06 22:51:38.172 UTC [9814] WARNING: there is no transaction in progress
2026-08-06 22:51:38,797 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-06 22:51:38,797 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQja9Us/build/release/bin/postgresql.jar:/tmp/dist-test-taskQja9Us/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.218.212:56393/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-06 22:51:39,015 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-06 22:51:39,015 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQja9Us/build/release/bin/postgresql.jar:/tmp/dist-test-taskQja9Us/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.218.212:56393/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 22:51:39,270 [I] Table x_portal_user already exists in database 'ranger'
2026-08-06 22:51:39,270 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQja9Us/build/release/bin/postgresql.jar:/tmp/dist-test-taskQja9Us/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.218.212:56393/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 22:51:39,535 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-06 22:51:39,535 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQja9Us/build/release/bin/postgresql.jar:/tmp/dist-test-taskQja9Us/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.218.212:56393/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 22:51:39,792 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQja9Us/build/release/bin/postgresql.jar:/tmp/dist-test-taskQja9Us/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.218.212:56393/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 22:51:40,044 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQja9Us/build/release/bin/postgresql.jar:/tmp/dist-test-taskQja9Us/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.218.212:56393/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-1xrh.c.gcp-upstream.internal';"
2026-08-06 22:51:40,267 [I] CORE_DB_SCHEMA import status has been updated
2026-08-06 22:51:40,268 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQja9Us/build/release/bin/postgresql.jar:/tmp/dist-test-taskQja9Us/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.218.212:56393/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-06 22:51:40,516 [I] DB_PATCHES have already been applied
I20260806 22:51:40.521721 29547 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/ranger-admin:/tmp/dist-test-taskQja9Us/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskQja9Us/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskQja9Us/build/release/bin/postgresql.jar:/tmp/dist-test-taskQja9Us/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260806 22:51:40.521782 29547 mini_ranger.cc:193] Using host: 127.28.218.212
I20260806 22:51:40.525523 29547 mini_ranger.cc:227] Ranger admin URL: http://127.28.218.212:52171
Aug 06, 2026 10:51:40 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 10:51:40 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 10:51:40 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/ranger-admin/ews/webapp
Aug 06, 2026 10:51:40 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 06, 2026 10:51:40 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 06, 2026 10:51:41 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/ranger-admin/ews/webapp].
Aug 06, 2026 10:51:41 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260806 22:51:41.642671 29547 mini_ranger.cc:160] Time spent starting Ranger: real 8.345s user 0.000s sys 0.010s
Aug 06, 2026 10:52:01 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 06, 2026 10:52:01 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 06, 2026 10:52:01 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 06, 2026 10:52:01 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 06, 2026 10:52:01 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 06, 2026 10:52:01 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 06, 2026 10:52:01 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 06, 2026 10:52:01 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 06, 2026 10:52:01 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 06, 2026 10:52:01 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 06, 2026 10:52:01 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 06, 2026 10:52:01 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 06, 2026 10:52:01 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 06, 2026 10:52:01 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 06, 2026 10:52:01 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 06, 2026 10:52:01 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 06, 2026 10:52:01 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 06, 2026 10:52:02 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260806 22:52:03.303994 29547 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.28.218.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.28.218.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.28.218.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/krb5kdc/rangerkms_127.28.218.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.28.218.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/krb5kdc/rangerkms_127.28.218.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.28.218.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.28.218.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.28.218.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/krb5kdc/HTTP_127.28.218.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.28.218.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/krb5kdc/HTTP_127.28.218.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 06 22:52:03 dist-test-slave-1xrh krb5kdc[9589](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056723, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260806 22:52:03.395447 29547 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260806 22:52:03.404731 29547 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260806 22:52:04.283421 29547 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260806 22:52:04.283512 29547 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/ranger-kms
I20260806 22:52:04.283531 29547 mini_ranger_kms.cc:227] Using postgres at 127.28.218.212:56393
2026-08-06 22:52:04,461 [I] DB FLAVOR :POSTGRES
2026-08-06 22:52:04,461 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 22:52:04,461 [I] Checking connection
2026-08-06 22:52:04,461 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQja9Us/build/release/bin/postgresql.jar:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.218.212:56393/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-06 22:52:04,682 [I] connection success
2026-08-06 22:52:04,682 [I] --------- Verifying Ranger DB tables ---------
2026-08-06 22:52:04,682 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-06 22:52:04,682 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQja9Us/build/release/bin/postgresql.jar:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.218.212:56393/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-06 22:52:04,934 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-06 22:52:04,934 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 22:52:04,934 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-06 22:52:04,934 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQja9Us/build/release/bin/postgresql.jar:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.218.212:56393/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-06 22:52:05,181 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260806 22:52:05.649820 29547 mini_ranger_kms.cc:310] Created kms service
I20260806 22:52:05.848628 29547 mini_ranger_kms.cc:326] Created kudu user
I20260806 22:52:05.891017 29547 mini_ranger_kms.cc:343] Created rangerkms user
Aug 06 22:52:05 dist-test-slave-1xrh krb5kdc[9589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056723, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.28.218.212@KRBTEST.COM
I20260806 22:52:06.247577 29547 mini_ranger_kms.cc:384] Added ranger policy
I20260806 22:52:06.247813 29547 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/ranger-kms:/tmp/dist-test-taskQja9Us/build/release/bin/postgresql.jar:/tmp/dist-test-taskQja9Us/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskQja9Us/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260806 22:52:06.247841 29547 mini_ranger_kms.cc:240] Using host: 127.28.218.212
I20260806 22:52:06.251919 29547 mini_ranger_kms.cc:276] Ranger KMS PID: 10006
I20260806 22:52:06.252004 29547 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.28.218.212:53613
Aug 06, 2026 10:52:07 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 10:52:07 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 10:52:07 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 06, 2026 10:52:07 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 06, 2026 10:52:07 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/ranger-kms/ews/webapp].
Aug 06, 2026 10:52:07 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260806 22:52:08.561017 29547 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.166s user 0.000s sys 0.161s
I20260806 22:52:08.561125 29547 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260806 22:52:08.561177 29547 mini_ranger_kms.cc:401] 127.28.218.212:53613/kms/v1/keys
Aug 06 22:52:08 dist-test-slave-1xrh krb5kdc[9589](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786056728, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.28.218.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 06 22:52:08 dist-test-slave-1xrh krb5kdc[9589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056728, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.28.218.212@KRBTEST.COM for HTTP/127.28.218.212@KRBTEST.COM
Aug 06 22:52:09 dist-test-slave-1xrh krb5kdc[9589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056728, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.28.218.212@KRBTEST.COM for HTTP/127.28.218.212@KRBTEST.COM
Aug 06 22:52:11 dist-test-slave-1xrh krb5kdc[9589](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056731, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.28.218.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.218.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.218.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab.
Entry for principal kudu/127.28.218.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab.
I20260806 22:52:11.529424 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.218.254:38697
--webserver_interface=127.28.218.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab
--principal=kudu/127.28.218.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.218.212:41891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.218.254:38697
--ranger_config_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.28.218.212:53613/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:52:11.612591 10061 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:52:11.612788 10061 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:52:11.612810 10061 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:52:11.613987 10061 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:52:11.614029 10061 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:52:11.614043 10061 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:52:11.614056 10061 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:52:11.614068 10061 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:52:11.615656 10061 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:41891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/master-0/wal
--ranger_kms_url=127.28.218.212:53613/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.28.218.254:38697
--ranger_config_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab
--principal=kudu/127.28.218.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.218.254:38697
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.218.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10061
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:52:11.615888 10061 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:52:11.616114 10061 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:52:11.620939 10067 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:52:11.620930 10069 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:52:11.621158 10061 server_base.cc:1034] running on GCE node
W20260806 22:52:11.620930 10066 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:52:11.621448 10061 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:52:11.621726 10061 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:52:11.622887 10061 hybrid_clock.cc:648] HybridClock initialized: now 1786056731622876 us; error 33 us; skew 500 ppm
Aug 06 22:52:11 dist-test-slave-1xrh krb5kdc[9589](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056731, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:52:11.629930 10061 init.cc:377] Logged in from keytab as kudu/127.28.218.254@KRBTEST.COM (short username kudu)
I20260806 22:52:11.630434 10061 webserver.cc:460] Webserver started at http://127.28.218.254:40265/ using document root <none> and password file <none>
I20260806 22:52:11.630676 10061 fs_manager.cc:359] Metadata directory not provided
I20260806 22:52:11.630731 10061 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:52:11.630874 10061 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 22:52:11 dist-test-slave-1xrh krb5kdc[9589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056731, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.254@KRBTEST.COM for HTTP/127.28.218.212@KRBTEST.COM
I20260806 22:52:11.675444 10061 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/master-0/data/instance:
uuid: "96cc599743cd4165823d9f6a6954420f"
format_stamp: "Formatted at 2026-08-06 22:52:11 on dist-test-slave-1xrh"
server_key: "45032f4fbb2836a8ee9ad1973aaf9b0d"
server_key_iv: "b37b32eafdca89b9585d8cb8d05ebb38"
server_key_version: "kuduclusterkey@0"
I20260806 22:52:11.675982 10061 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/master-0/wal/instance:
uuid: "96cc599743cd4165823d9f6a6954420f"
format_stamp: "Formatted at 2026-08-06 22:52:11 on dist-test-slave-1xrh"
server_key: "45032f4fbb2836a8ee9ad1973aaf9b0d"
server_key_iv: "b37b32eafdca89b9585d8cb8d05ebb38"
server_key_version: "kuduclusterkey@0"
I20260806 22:52:11.677551 10061 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 22:52:11.688555 10076 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:52:11.688755 10061 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 22:52:11.688854 10061 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/master-0/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/master-0/wal
uuid: "96cc599743cd4165823d9f6a6954420f"
format_stamp: "Formatted at 2026-08-06 22:52:11 on dist-test-slave-1xrh"
server_key: "45032f4fbb2836a8ee9ad1973aaf9b0d"
server_key_iv: "b37b32eafdca89b9585d8cb8d05ebb38"
server_key_version: "kuduclusterkey@0"
I20260806 22:52:11.688952 10061 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:52:11.702487 10061 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:52:11.704471 10061 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:52:11.704628 10061 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:52:11.709082 10061 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.254:38697
I20260806 22:52:11.709121 10128 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.254:38697 every 8 connection(s)
I20260806 22:52:11.709455 10061 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
I20260806 22:52:11.710393 10129 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:52:11.713822 10129 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96cc599743cd4165823d9f6a6954420f: Bootstrap starting.
I20260806 22:52:11.714600 10129 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 96cc599743cd4165823d9f6a6954420f: Neither blocks nor log segments found. Creating new log.
I20260806 22:52:11.714879 10129 log.cc:824] T 00000000000000000000000000000000 P 96cc599743cd4165823d9f6a6954420f: Log is configured to *not* fsync() on all Append() calls
I20260806 22:52:11.714994 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 10061
I20260806 22:52:11.715113 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/master-0/wal/instance
I20260806 22:52:11.715323 29547 external_mini_cluster.cc:1428] Setting key 6f29056591021c82c4b0fbbd1085b127
I20260806 22:52:11.715821 10129 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96cc599743cd4165823d9f6a6954420f: No bootstrap required, opened a new log
I20260806 22:52:11.718231 10129 raft_consensus.cc:348] T 00000000000000000000000000000000 P 96cc599743cd4165823d9f6a6954420f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96cc599743cd4165823d9f6a6954420f" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 38697 } }
I20260806 22:52:11.718389 10129 raft_consensus.cc:374] T 00000000000000000000000000000000 P 96cc599743cd4165823d9f6a6954420f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:52:11.718416 10129 raft_consensus.cc:729] T 00000000000000000000000000000000 P 96cc599743cd4165823d9f6a6954420f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96cc599743cd4165823d9f6a6954420f, State: Initialized, Role: FOLLOWER
I20260806 22:52:11.718578 10129 consensus_queue.cc:260] T 00000000000000000000000000000000 P 96cc599743cd4165823d9f6a6954420f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96cc599743cd4165823d9f6a6954420f" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 38697 } }
I20260806 22:52:11.718660 10129 raft_consensus.cc:388] T 00000000000000000000000000000000 P 96cc599743cd4165823d9f6a6954420f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:52:11.718705 10129 raft_consensus.cc:482] T 00000000000000000000000000000000 P 96cc599743cd4165823d9f6a6954420f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:52:11.718755 10129 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 96cc599743cd4165823d9f6a6954420f [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:52:11.719408 10129 raft_consensus.cc:504] T 00000000000000000000000000000000 P 96cc599743cd4165823d9f6a6954420f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96cc599743cd4165823d9f6a6954420f" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 38697 } }
I20260806 22:52:11.719537 10129 leader_election.cc:302] T 00000000000000000000000000000000 P 96cc599743cd4165823d9f6a6954420f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 96cc599743cd4165823d9f6a6954420f; no voters:
I20260806 22:52:11.719733 10129 leader_election.cc:288] T 00000000000000000000000000000000 P 96cc599743cd4165823d9f6a6954420f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:52:11.719766 10134 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 96cc599743cd4165823d9f6a6954420f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:52:11.719908 10134 raft_consensus.cc:686] T 00000000000000000000000000000000 P 96cc599743cd4165823d9f6a6954420f [term 1 LEADER]: Becoming Leader. State: Replica: 96cc599743cd4165823d9f6a6954420f, State: Running, Role: LEADER
I20260806 22:52:11.720031 10129 sys_catalog.cc:564] T 00000000000000000000000000000000 P 96cc599743cd4165823d9f6a6954420f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:52:11.720079 10134 consensus_queue.cc:237] T 00000000000000000000000000000000 P 96cc599743cd4165823d9f6a6954420f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96cc599743cd4165823d9f6a6954420f" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 38697 } }
I20260806 22:52:11.720492 10134 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96cc599743cd4165823d9f6a6954420f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 96cc599743cd4165823d9f6a6954420f. Latest consensus state: current_term: 1 leader_uuid: "96cc599743cd4165823d9f6a6954420f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96cc599743cd4165823d9f6a6954420f" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 38697 } } }
I20260806 22:52:11.720498 10135 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96cc599743cd4165823d9f6a6954420f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "96cc599743cd4165823d9f6a6954420f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96cc599743cd4165823d9f6a6954420f" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 38697 } } }
I20260806 22:52:11.720701 10134 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96cc599743cd4165823d9f6a6954420f [sys.catalog]: This master's current role is: LEADER
I20260806 22:52:11.720714 10135 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96cc599743cd4165823d9f6a6954420f [sys.catalog]: This master's current role is: LEADER
I20260806 22:52:11.721036 10143 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:52:11.721940 10143 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:52:11.722661 10129 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
Aug 06 22:52:11 dist-test-slave-1xrh krb5kdc[9589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056731, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.218.254@KRBTEST.COM
I20260806 22:52:11.727891 10143 catalog_manager.cc:1347] Generated new cluster ID: a987738e1cb045c7aa3780dc5ff8ec3a
I20260806 22:52:11.728076 10143 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:52:11.730706 10133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:11.715843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56546 (local address 127.28.218.254:38697)
0806 22:52:11.716008 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0806 22:52:11.716016 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:52:11.716175 (+ 159us) server_negotiation.cc:407] Connection header received
0806 22:52:11.716966 (+ 791us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:52:11.716973 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:52:11.717029 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:52:11.717168 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:52:11.717811 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:11.718948 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:11.719866 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:11.720565 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:11.720648 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:52:11.724970 (+ 4322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:52:11.725034 (+ 64us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:52:11.725035 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:52:11.725171 (+ 136us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:52:11.728526 (+ 3355us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:52:11.728954 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:52:11.728957 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:52:11.728958 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:52:11.729089 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:52:11.729298 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:52:11.729299 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:52:11.729300 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:52:11.729651 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:52:11.729823 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0806 22:52:11.729919 (+ 96us) server_negotiation.cc:299] Negotiation successful
0806 22:52:11.730349 (+ 430us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":82,"threads_started":1}
I20260806 22:52:11.736954 10143 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:52:11.737568 10143 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:52:11.744565 10143 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 96cc599743cd4165823d9f6a6954420f: Generated new TSK 0
I20260806 22:52:11.744756 10143 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.28.218.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.218.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.218.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.193.keytab.
Entry for principal kudu/127.28.218.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.193.keytab.
I20260806 22:52:12.455185 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.28.218.212:53613/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.218.193:0
--local_ip_for_outbound_sockets=127.28.218.193
--webserver_interface=127.28.218.193
--webserver_port=0
--tserver_master_addrs=127.28.218.254:38697
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.193.keytab
--principal=kudu/127.28.218.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.218.212:41891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:52:12.554870 10177 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:52:12.555089 10177 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:52:12.555125 10177 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:52:12.556275 10177 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:52:12.556346 10177 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:52:12.556391 10177 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.218.193
I20260806 22:52:12.557811 10177 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:41891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.28.218.212:53613/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.193.keytab
--principal=kudu/127.28.218.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.218.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.218.193
--webserver_port=0
--tserver_master_addrs=127.28.218.254:38697
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10177
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.218.193
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:52:12.558053 10177 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:52:12.558266 10177 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:52:12.563571 10183 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:52:12.563726 10182 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:52:12.563627 10185 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:52:12.564028 10177 server_base.cc:1034] running on GCE node
I20260806 22:52:12.564256 10177 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:52:12.564482 10177 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:52:12.565662 10177 hybrid_clock.cc:648] HybridClock initialized: now 1786056732565633 us; error 45 us; skew 500 ppm
Aug 06 22:52:12 dist-test-slave-1xrh krb5kdc[9589](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056732, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:52:12.568882 10177 init.cc:377] Logged in from keytab as kudu/127.28.218.193@KRBTEST.COM (short username kudu)
I20260806 22:52:12.569355 10177 webserver.cc:460] Webserver started at http://127.28.218.193:45739/ using document root <none> and password file <none>
I20260806 22:52:12.569571 10177 fs_manager.cc:359] Metadata directory not provided
I20260806 22:52:12.569633 10177 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:52:12.569758 10177 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 22:52:12 dist-test-slave-1xrh krb5kdc[9589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056732, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.193@KRBTEST.COM for HTTP/127.28.218.212@KRBTEST.COM
I20260806 22:52:12.581964 10177 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/ts-0/data/instance:
uuid: "3a45c988f3ee4979a63b75d595bea1b7"
format_stamp: "Formatted at 2026-08-06 22:52:12 on dist-test-slave-1xrh"
server_key: "f57f4600077051eb3c9089f7ebaf7a70"
server_key_iv: "a4c9fe01e18b730346a3ab342c41d1c9"
server_key_version: "kuduclusterkey@0"
I20260806 22:52:12.582357 10177 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/ts-0/wal/instance:
uuid: "3a45c988f3ee4979a63b75d595bea1b7"
format_stamp: "Formatted at 2026-08-06 22:52:12 on dist-test-slave-1xrh"
server_key: "f57f4600077051eb3c9089f7ebaf7a70"
server_key_iv: "a4c9fe01e18b730346a3ab342c41d1c9"
server_key_version: "kuduclusterkey@0"
I20260806 22:52:12.583869 10177 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:52:12.597838 10194 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:52:12.598062 10177 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:52:12.598158 10177 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/ts-0/wal
uuid: "3a45c988f3ee4979a63b75d595bea1b7"
format_stamp: "Formatted at 2026-08-06 22:52:12 on dist-test-slave-1xrh"
server_key: "f57f4600077051eb3c9089f7ebaf7a70"
server_key_iv: "a4c9fe01e18b730346a3ab342c41d1c9"
server_key_version: "kuduclusterkey@0"
I20260806 22:52:12.598248 10177 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:52:12.645118 10177 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:52:12.647702 10177 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:52:12.647918 10177 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:52:12.648231 10177 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:52:12.648658 10177 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:52:12.648749 10177 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:52:12.648869 10177 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:52:12.648950 10177 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:52:12.656559 10177 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.193:46773
I20260806 22:52:12.657071 10177 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/ts-0/data/info.pb
I20260806 22:52:12.661656 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 10177
I20260806 22:52:12.661772 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/ts-0/wal/instance
I20260806 22:52:12.662004 29547 external_mini_cluster.cc:1428] Setting key df556c2a2d5a7bc116baa3ddc185505a
I20260806 22:52:12.659783 10312 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.193:46773 every 8 connection(s)
Aug 06 22:52:12 dist-test-slave-1xrh krb5kdc[9589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056732, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.193@KRBTEST.COM for kudu/127.28.218.254@KRBTEST.COM
I20260806 22:52:12.674445 10316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:12.659330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:38697 (local address 127.28.218.193:45791)
0806 22:52:12.663241 (+ 3911us) negotiation.cc:107] Waiting for socket to connect
0806 22:52:12.663291 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 22:52:12.663746 (+ 455us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:52:12.664697 (+ 951us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:52:12.664702 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:52:12.664975 (+ 273us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:52:12.665262 (+ 287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:52:12.665270 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:52:12.666360 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:52:12.666361 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:52:12.667698 (+ 1337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:52:12.667701 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:52:12.668276 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:52:12.668278 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:52:12.668422 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:52:12.669055 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:52:12.669065 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:52:12.670744 (+ 1679us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:52:12.672767 (+ 2023us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:52:12.672773 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:52:12.672776 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:52:12.673075 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:52:12.673305 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:52:12.673305 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:52:12.673306 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:52:12.673472 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:52:12.673776 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:52:12.673782 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:52:12.673944 (+ 162us) client_negotiation.cc:769] Sending connection context
0806 22:52:12.674038 (+ 94us) client_negotiation.cc:240] Negotiation successful
0806 22:52:12.674191 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3860,"thread_start_us":81,"threads_started":1}
I20260806 22:52:12.675352 10317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:12.659616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.193:45791 (local address 127.28.218.254:38697)
0806 22:52:12.664406 (+ 4790us) server_negotiation.cc:206] Beginning negotiation
0806 22:52:12.664408 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:52:12.664416 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:52:12.664437 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:52:12.664437 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:52:12.664447 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:52:12.664552 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:52:12.665380 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:12.666227 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:12.667830 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:12.668159 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:12.668235 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:52:12.670866 (+ 2631us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:52:12.670867 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:52:12.670867 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:52:12.670875 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:52:12.672676 (+ 1801us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:52:12.673171 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:52:12.673172 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:52:12.673172 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:52:12.673236 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:52:12.673554 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:52:12.673555 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:52:12.673555 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:52:12.673697 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:52:12.675040 (+ 1343us) server_negotiation.cc:1035] Waiting for connection context
0806 22:52:12.675087 (+ 47us) server_negotiation.cc:299] Negotiation successful
0806 22:52:12.675201 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4762,"thread_start_us":78,"threads_started":1}
WARNING: no policy specified for kudu/127.28.218.194@KRBTEST.COM; defaulting to no policy
I20260806 22:52:12.675992 10313 heartbeater.cc:347] Connected to a master server at 127.28.218.254:38697
I20260806 22:52:12.676083 10313 heartbeater.cc:464] Registering TS with master...
I20260806 22:52:12.676262 10313 heartbeater.cc:511] Master 127.28.218.254:38697 requested a full tablet report, sending...
I20260806 22:52:12.676852 10093 ts_manager.cc:194] Registered new tserver with Master: 3a45c988f3ee4979a63b75d595bea1b7 (127.28.218.193:46773)
I20260806 22:52:12.677682 10093 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.218.193@KRBTEST.COM'} at 127.28.218.193:45791
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.218.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.218.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.194.keytab.
Entry for principal kudu/127.28.218.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.194.keytab.
I20260806 22:52:12.697885 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.28.218.212:53613/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.218.194:0
--local_ip_for_outbound_sockets=127.28.218.194
--webserver_interface=127.28.218.194
--webserver_port=0
--tserver_master_addrs=127.28.218.254:38697
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.194.keytab
--principal=kudu/127.28.218.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.218.212:41891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 22:52:12 dist-test-slave-1xrh krb5kdc[9589](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786056732, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260806 22:52:12.806869 10322 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:52:12.807060 10322 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:52:12.807083 10322 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:52:12.808255 10322 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:52:12.808310 10322 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:52:12.808350 10322 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.218.194
I20260806 22:52:12.809823 10322 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:41891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.28.218.212:53613/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.194.keytab
--principal=kudu/127.28.218.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.218.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.218.194
--webserver_port=0
--tserver_master_addrs=127.28.218.254:38697
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10322
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.218.194
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:52:12.810056 10322 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:52:12.810281 10322 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:52:12.815538 10331 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:52:12.815543 10327 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:52:12.815699 10328 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:52:12.816237 10322 server_base.cc:1034] running on GCE node
I20260806 22:52:12.816442 10322 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:52:12.816737 10322 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:52:12.817919 10322 hybrid_clock.cc:648] HybridClock initialized: now 1786056732817895 us; error 40 us; skew 500 ppm
Aug 06 22:52:12 dist-test-slave-1xrh krb5kdc[9589](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056732, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:52:12.821256 10322 init.cc:377] Logged in from keytab as kudu/127.28.218.194@KRBTEST.COM (short username kudu)
I20260806 22:52:12.821698 10322 webserver.cc:460] Webserver started at http://127.28.218.194:33809/ using document root <none> and password file <none>
I20260806 22:52:12.821923 10322 fs_manager.cc:359] Metadata directory not provided
I20260806 22:52:12.821969 10322 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:52:12.822098 10322 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 22:52:12 dist-test-slave-1xrh krb5kdc[9589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056732, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.194@KRBTEST.COM for HTTP/127.28.218.212@KRBTEST.COM
I20260806 22:52:12.834918 10322 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/ts-1/data/instance:
uuid: "b4f6bf590eb347008b1d9ca0ed128df8"
format_stamp: "Formatted at 2026-08-06 22:52:12 on dist-test-slave-1xrh"
server_key: "e940eaf14b1fd9d993615a66117863d3"
server_key_iv: "c0084e7787eb780a16d5aa5f8f8f3ef2"
server_key_version: "kuduclusterkey@0"
I20260806 22:52:12.835438 10322 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/ts-1/wal/instance:
uuid: "b4f6bf590eb347008b1d9ca0ed128df8"
format_stamp: "Formatted at 2026-08-06 22:52:12 on dist-test-slave-1xrh"
server_key: "e940eaf14b1fd9d993615a66117863d3"
server_key_iv: "c0084e7787eb780a16d5aa5f8f8f3ef2"
server_key_version: "kuduclusterkey@0"
I20260806 22:52:12.837039 10322 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:52:12.856964 10339 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:52:12.857196 10322 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.000s
I20260806 22:52:12.857306 10322 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/ts-1/wal
uuid: "b4f6bf590eb347008b1d9ca0ed128df8"
format_stamp: "Formatted at 2026-08-06 22:52:12 on dist-test-slave-1xrh"
server_key: "e940eaf14b1fd9d993615a66117863d3"
server_key_iv: "c0084e7787eb780a16d5aa5f8f8f3ef2"
server_key_version: "kuduclusterkey@0"
I20260806 22:52:12.857398 10322 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:52:12.878922 10322 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:52:12.881330 10322 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:52:12.881486 10322 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:52:12.881742 10322 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:52:12.882097 10322 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:52:12.882144 10322 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:52:12.882174 10322 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:52:12.882241 10322 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:52:12.890571 10322 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.194:38293
I20260806 22:52:12.891009 10322 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/ts-1/data/info.pb
I20260806 22:52:12.894390 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 10322
I20260806 22:52:12.894505 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/ts-1/wal/instance
I20260806 22:52:12.894726 29547 external_mini_cluster.cc:1428] Setting key c36ac0db6135f3f3b94b704c3b5249f9
Aug 06 22:52:12 dist-test-slave-1xrh krb5kdc[9589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056732, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.194@KRBTEST.COM for kudu/127.28.218.254@KRBTEST.COM
I20260806 22:52:12.900631 10459 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.194:38293 every 8 connection(s)
I20260806 22:52:12.902835 10317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:12.891764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.194:54897 (local address 127.28.218.254:38697)
0806 22:52:12.891808 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 22:52:12.891810 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:52:12.892079 (+ 269us) server_negotiation.cc:407] Connection header received
0806 22:52:12.892734 (+ 655us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:52:12.892736 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:52:12.892753 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:52:12.892785 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:52:12.893604 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:12.894403 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:12.896347 (+ 1944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:12.896682 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:12.896734 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:52:12.899423 (+ 2689us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:52:12.899425 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:52:12.899425 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:52:12.899433 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:52:12.901147 (+ 1714us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:52:12.901677 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:52:12.901678 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:52:12.901679 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:52:12.901738 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:52:12.902041 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:52:12.902042 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:52:12.902042 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:52:12.902179 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:52:12.902227 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 22:52:12.902624 (+ 397us) server_negotiation.cc:299] Negotiation successful
0806 22:52:12.902725 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 22:52:12.903558 10462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:12.891773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:38697 (local address 127.28.218.194:54897)
0806 22:52:12.891928 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0806 22:52:12.891975 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 22:52:12.892605 (+ 630us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:52:12.892965 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:52:12.892969 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:52:12.893216 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:52:12.893488 (+ 272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:52:12.893494 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:52:12.894523 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:52:12.894524 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:52:12.896214 (+ 1690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:52:12.896217 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:52:12.896844 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:52:12.896845 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:52:12.896991 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:52:12.897586 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:52:12.897596 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:52:12.899291 (+ 1695us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:52:12.901281 (+ 1990us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:52:12.901287 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:52:12.901289 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:52:12.901581 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:52:12.901814 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:52:12.901815 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:52:12.901815 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:52:12.901946 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:52:12.902280 (+ 334us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:52:12.902286 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:52:12.902454 (+ 168us) client_negotiation.cc:769] Sending connection context
0806 22:52:12.903158 (+ 704us) client_negotiation.cc:240] Negotiation successful
0806 22:52:12.903334 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":78,"threads_started":1}
WARNING: no policy specified for kudu/127.28.218.195@KRBTEST.COM; defaulting to no policy
I20260806 22:52:12.911060 10460 heartbeater.cc:347] Connected to a master server at 127.28.218.254:38697
I20260806 22:52:12.911222 10460 heartbeater.cc:464] Registering TS with master...
I20260806 22:52:12.911464 10460 heartbeater.cc:511] Master 127.28.218.254:38697 requested a full tablet report, sending...
I20260806 22:52:12.911898 10093 ts_manager.cc:194] Registered new tserver with Master: b4f6bf590eb347008b1d9ca0ed128df8 (127.28.218.194:38293)
I20260806 22:52:12.912393 10093 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.218.194@KRBTEST.COM'} at 127.28.218.194:54897
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.218.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.218.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.195.keytab.
Entry for principal kudu/127.28.218.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.195.keytab.
I20260806 22:52:12.934864 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.28.218.212:53613/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.218.195:0
--local_ip_for_outbound_sockets=127.28.218.195
--webserver_interface=127.28.218.195
--webserver_port=0
--tserver_master_addrs=127.28.218.254:38697
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.195.keytab
--principal=kudu/127.28.218.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.218.212:41891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:52:13.054996 10465 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:52:13.055282 10465 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:52:13.055321 10465 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:52:13.056988 10465 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:52:13.057116 10465 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:52:13.057210 10465 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.218.195
I20260806 22:52:13.059468 10465 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:41891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.28.218.212:53613/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.195.keytab
--principal=kudu/127.28.218.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.218.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.218.195
--webserver_port=0
--tserver_master_addrs=127.28.218.254:38697
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10465
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.218.195
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:52:13.059902 10465 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:52:13.060220 10465 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:52:13.067164 10470 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:52:13.067265 10473 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:52:13.067514 10471 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:52:13.068225 10465 server_base.cc:1034] running on GCE node
I20260806 22:52:13.068454 10465 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:52:13.068707 10465 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:52:13.069859 10465 hybrid_clock.cc:648] HybridClock initialized: now 1786056733069835 us; error 39 us; skew 500 ppm
Aug 06 22:52:13 dist-test-slave-1xrh krb5kdc[9589](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056733, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:52:13.073426 10465 init.cc:377] Logged in from keytab as kudu/127.28.218.195@KRBTEST.COM (short username kudu)
I20260806 22:52:13.073992 10465 webserver.cc:460] Webserver started at http://127.28.218.195:35391/ using document root <none> and password file <none>
I20260806 22:52:13.074306 10465 fs_manager.cc:359] Metadata directory not provided
I20260806 22:52:13.074420 10465 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:52:13.074589 10465 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 22:52:13 dist-test-slave-1xrh krb5kdc[9589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056733, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.195@KRBTEST.COM for HTTP/127.28.218.212@KRBTEST.COM
I20260806 22:52:13.088693 10465 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/ts-2/data/instance:
uuid: "1fde50869d5b47018effa45a1417d1d1"
format_stamp: "Formatted at 2026-08-06 22:52:13 on dist-test-slave-1xrh"
server_key: "1839b6a86b3685cb7fd550f56b88a018"
server_key_iv: "8df9332bff6dfdb1f374f9d97babc2c2"
server_key_version: "kuduclusterkey@0"
I20260806 22:52:13.089252 10465 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/ts-2/wal/instance:
uuid: "1fde50869d5b47018effa45a1417d1d1"
format_stamp: "Formatted at 2026-08-06 22:52:13 on dist-test-slave-1xrh"
server_key: "1839b6a86b3685cb7fd550f56b88a018"
server_key_iv: "8df9332bff6dfdb1f374f9d97babc2c2"
server_key_version: "kuduclusterkey@0"
I20260806 22:52:13.095638 10465 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.000s
I20260806 22:52:13.109483 10480 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:52:13.109695 10465 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 22:52:13.109789 10465 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/ts-2/wal
uuid: "1fde50869d5b47018effa45a1417d1d1"
format_stamp: "Formatted at 2026-08-06 22:52:13 on dist-test-slave-1xrh"
server_key: "1839b6a86b3685cb7fd550f56b88a018"
server_key_iv: "8df9332bff6dfdb1f374f9d97babc2c2"
server_key_version: "kuduclusterkey@0"
I20260806 22:52:13.109866 10465 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:52:13.139532 10465 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:52:13.142310 10465 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:52:13.142536 10465 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:52:13.142849 10465 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:52:13.143306 10465 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:52:13.143409 10465 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:52:13.143481 10465 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:52:13.143539 10465 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:52:13.157795 10465 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.195:33451
I20260806 22:52:13.158322 10465 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/ts-2/data/info.pb
I20260806 22:52:13.163646 10596 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.195:33451 every 8 connection(s)
I20260806 22:52:13.166013 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 10465
I20260806 22:52:13.166139 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/ts-2/wal/instance
I20260806 22:52:13.166340 29547 external_mini_cluster.cc:1428] Setting key 32139c82411cafe155ff7adf41a28a32
Aug 06 22:52:13 dist-test-slave-1xrh krb5kdc[9589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056733, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.195@KRBTEST.COM for kudu/127.28.218.254@KRBTEST.COM
I20260806 22:52:13.172885 10317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:13.159829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.195:43351 (local address 127.28.218.254:38697)
0806 22:52:13.159873 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 22:52:13.159875 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:52:13.162213 (+ 2338us) server_negotiation.cc:407] Connection header received
0806 22:52:13.162732 (+ 519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:52:13.162733 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:52:13.162747 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:52:13.162792 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:52:13.163927 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:13.164736 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:13.166454 (+ 1718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:13.166732 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:13.166770 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:52:13.169307 (+ 2537us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:52:13.169309 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:52:13.169309 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:52:13.169317 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:52:13.171058 (+ 1741us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:52:13.171542 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:52:13.171543 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:52:13.171543 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:52:13.171609 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:52:13.171943 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:52:13.171944 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:52:13.171944 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:52:13.172078 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:52:13.172135 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 22:52:13.172684 (+ 549us) server_negotiation.cc:299] Negotiation successful
0806 22:52:13.172779 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 22:52:13.173619 10599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:13.159446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:38697 (local address 127.28.218.195:43351)
0806 22:52:13.162086 (+ 2640us) negotiation.cc:107] Waiting for socket to connect
0806 22:52:13.162124 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 22:52:13.162630 (+ 506us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:52:13.162869 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:52:13.162873 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:52:13.163065 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:52:13.163404 (+ 339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:52:13.163411 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:52:13.164862 (+ 1451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:52:13.164863 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:52:13.166313 (+ 1450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:52:13.166317 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:52:13.166874 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:52:13.166875 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:52:13.167002 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:52:13.167586 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:52:13.167596 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:52:13.169204 (+ 1608us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:52:13.171181 (+ 1977us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:52:13.171188 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:52:13.171190 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:52:13.171460 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:52:13.171695 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:52:13.171696 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:52:13.171697 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:52:13.171852 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:52:13.172346 (+ 494us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:52:13.172352 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:52:13.172512 (+ 160us) client_negotiation.cc:769] Sending connection context
0806 22:52:13.173227 (+ 715us) client_negotiation.cc:240] Negotiation successful
0806 22:52:13.173395 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2603,"thread_start_us":184,"threads_started":1}
I20260806 22:52:13.174059 10597 heartbeater.cc:347] Connected to a master server at 127.28.218.254:38697
I20260806 22:52:13.174165 10597 heartbeater.cc:464] Registering TS with master...
I20260806 22:52:13.174345 10597 heartbeater.cc:511] Master 127.28.218.254:38697 requested a full tablet report, sending...
I20260806 22:52:13.174721 10093 ts_manager.cc:194] Registered new tserver with Master: 1fde50869d5b47018effa45a1417d1d1 (127.28.218.195:33451)
I20260806 22:52:13.175168 10093 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.218.195@KRBTEST.COM'} at 127.28.218.195:43351
I20260806 22:52:13.177793 29547 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:52:13.178483 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 10177
I20260806 22:52:13.187542 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 10322
I20260806 22:52:13.197826 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 10465
I20260806 22:52:13.208163 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 10061
I20260806 22:52:13.216159 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.218.254:38697
--webserver_interface=127.28.218.254
--webserver_port=40265
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab
--principal=kudu/127.28.218.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.218.212:41891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.218.254:38697
--ranger_config_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.28.218.212:53613/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.28.218.212:53613/kms with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:52:13.297230 10600 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:52:13.297420 10600 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:52:13.297442 10600 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:52:13.298568 10600 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:52:13.298611 10600 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:52:13.298625 10600 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:52:13.298638 10600 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:52:13.298691 10600 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:52:13.300275 10600 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:41891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.28.218.212:53613/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.28.218.254:38697
--ranger_config_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab
--principal=kudu/127.28.218.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.218.254:38697
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.218.254
--webserver_port=40265
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10600
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:52:13.300499 10600 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:52:13.300714 10600 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:52:13.305296 10605 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:52:13.305362 10606 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:52:13.305382 10608 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:52:13.305548 10600 server_base.cc:1034] running on GCE node
I20260806 22:52:13.305768 10600 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:52:13.306000 10600 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:52:13.307166 10600 hybrid_clock.cc:648] HybridClock initialized: now 1786056733307144 us; error 33 us; skew 500 ppm
Aug 06 22:52:13 dist-test-slave-1xrh krb5kdc[9589](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056733, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:52:13.309594 10600 init.cc:377] Logged in from keytab as kudu/127.28.218.254@KRBTEST.COM (short username kudu)
I20260806 22:52:13.310006 10600 webserver.cc:460] Webserver started at http://127.28.218.254:40265/ using document root <none> and password file <none>
I20260806 22:52:13.310308 10600 fs_manager.cc:359] Metadata directory not provided
I20260806 22:52:13.310389 10600 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 22:52:13 dist-test-slave-1xrh krb5kdc[9589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056733, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.254@KRBTEST.COM for HTTP/127.28.218.212@KRBTEST.COM
I20260806 22:52:13.358741 10600 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:52:13.359583 10618 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:52:13.359778 10600 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 22:52:13.359887 10600 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/master-0/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/master-0/wal
uuid: "96cc599743cd4165823d9f6a6954420f"
format_stamp: "Formatted at 2026-08-06 22:52:11 on dist-test-slave-1xrh"
server_key: "45032f4fbb2836a8ee9ad1973aaf9b0d"
server_key_iv: "b37b32eafdca89b9585d8cb8d05ebb38"
server_key_version: "kuduclusterkey@0"
I20260806 22:52:13.360390 10600 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:52:13.381168 10600 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:52:13.383273 10600 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:52:13.383428 10600 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:52:13.387401 10600 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.254:38697
I20260806 22:52:13.387451 10675 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.254:38697 every 8 connection(s)
I20260806 22:52:13.387768 10600 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
I20260806 22:52:13.388859 10676 sys_catalog.cc:263] Verifying existing consensus state
I20260806 22:52:13.389473 10676 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96cc599743cd4165823d9f6a6954420f: Bootstrap starting.
I20260806 22:52:13.392141 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 10600
I20260806 22:52:13.392429 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.28.218.212:53613/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.218.193:46773
--local_ip_for_outbound_sockets=127.28.218.193
--tserver_master_addrs=127.28.218.254:38697
--webserver_port=45739
--webserver_interface=127.28.218.193
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.193.keytab
--principal=kudu/127.28.218.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.218.212:41891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.28.218.212:53613/kms with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:52:13.397464 10676 log.cc:824] T 00000000000000000000000000000000 P 96cc599743cd4165823d9f6a6954420f: Log is configured to *not* fsync() on all Append() calls
I20260806 22:52:13.399034 10676 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96cc599743cd4165823d9f6a6954420f: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 22:52:13.399403 10676 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96cc599743cd4165823d9f6a6954420f: Bootstrap complete.
I20260806 22:52:13.401813 10676 raft_consensus.cc:348] T 00000000000000000000000000000000 P 96cc599743cd4165823d9f6a6954420f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96cc599743cd4165823d9f6a6954420f" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 38697 } }
I20260806 22:52:13.402057 10676 raft_consensus.cc:729] T 00000000000000000000000000000000 P 96cc599743cd4165823d9f6a6954420f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96cc599743cd4165823d9f6a6954420f, State: Initialized, Role: FOLLOWER
I20260806 22:52:13.402223 10676 consensus_queue.cc:260] T 00000000000000000000000000000000 P 96cc599743cd4165823d9f6a6954420f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96cc599743cd4165823d9f6a6954420f" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 38697 } }
I20260806 22:52:13.402294 10676 raft_consensus.cc:388] T 00000000000000000000000000000000 P 96cc599743cd4165823d9f6a6954420f [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:52:13.402345 10676 raft_consensus.cc:482] T 00000000000000000000000000000000 P 96cc599743cd4165823d9f6a6954420f [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:52:13.402415 10676 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 96cc599743cd4165823d9f6a6954420f [term 1 FOLLOWER]: Advancing to term 2
I20260806 22:52:13.403411 10676 raft_consensus.cc:504] T 00000000000000000000000000000000 P 96cc599743cd4165823d9f6a6954420f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96cc599743cd4165823d9f6a6954420f" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 38697 } }
I20260806 22:52:13.403522 10676 leader_election.cc:302] T 00000000000000000000000000000000 P 96cc599743cd4165823d9f6a6954420f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 96cc599743cd4165823d9f6a6954420f; no voters:
I20260806 22:52:13.403702 10676 leader_election.cc:288] T 00000000000000000000000000000000 P 96cc599743cd4165823d9f6a6954420f [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 22:52:13.403734 10681 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 96cc599743cd4165823d9f6a6954420f [term 2 FOLLOWER]: Leader election won for term 2
I20260806 22:52:13.403877 10681 raft_consensus.cc:686] T 00000000000000000000000000000000 P 96cc599743cd4165823d9f6a6954420f [term 2 LEADER]: Becoming Leader. State: Replica: 96cc599743cd4165823d9f6a6954420f, State: Running, Role: LEADER
I20260806 22:52:13.403996 10676 sys_catalog.cc:564] T 00000000000000000000000000000000 P 96cc599743cd4165823d9f6a6954420f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:52:13.404039 10681 consensus_queue.cc:237] T 00000000000000000000000000000000 P 96cc599743cd4165823d9f6a6954420f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96cc599743cd4165823d9f6a6954420f" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 38697 } }
I20260806 22:52:13.404377 10688 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96cc599743cd4165823d9f6a6954420f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "96cc599743cd4165823d9f6a6954420f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96cc599743cd4165823d9f6a6954420f" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 38697 } } }
I20260806 22:52:13.404397 10689 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96cc599743cd4165823d9f6a6954420f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 96cc599743cd4165823d9f6a6954420f. Latest consensus state: current_term: 2 leader_uuid: "96cc599743cd4165823d9f6a6954420f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96cc599743cd4165823d9f6a6954420f" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 38697 } } }
I20260806 22:52:13.404513 10689 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96cc599743cd4165823d9f6a6954420f [sys.catalog]: This master's current role is: LEADER
I20260806 22:52:13.404469 10688 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96cc599743cd4165823d9f6a6954420f [sys.catalog]: This master's current role is: LEADER
I20260806 22:52:13.404775 10695 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:52:13.405334 10695 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:52:13.405491 10695 catalog_manager.cc:1259] Loaded cluster ID: a987738e1cb045c7aa3780dc5ff8ec3a
I20260806 22:52:13.405578 10695 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:52:13.405764 10676 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260806 22:52:13.411689 10695 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:52:13.411957 10695 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 96cc599743cd4165823d9f6a6954420f: Loaded TSK: 0
I20260806 22:52:13.412173 10695 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 22:52:13.508050 10679 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:52:13.508414 10679 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:52:13.508507 10679 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:52:13.510237 10679 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:52:13.510350 10679 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:52:13.510437 10679 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.218.193
I20260806 22:52:13.512588 10679 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:41891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.28.218.212:53613/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.193.keytab
--principal=kudu/127.28.218.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.218.193:46773
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.218.193
--webserver_port=45739
--tserver_master_addrs=127.28.218.254:38697
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10679
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.218.193
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:52:13.513109 10679 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:52:13.513468 10679 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:52:13.519953 10724 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:52:13.519927 10725 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:52:13.519942 10727 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:52:13.520516 10679 server_base.cc:1034] running on GCE node
I20260806 22:52:13.520716 10679 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:52:13.520995 10679 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:52:13.522173 10679 hybrid_clock.cc:648] HybridClock initialized: now 1786056733522158 us; error 28 us; skew 500 ppm
Aug 06 22:52:13 dist-test-slave-1xrh krb5kdc[9589](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056733, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:52:13.525511 10679 init.cc:377] Logged in from keytab as kudu/127.28.218.193@KRBTEST.COM (short username kudu)
I20260806 22:52:13.526037 10679 webserver.cc:460] Webserver started at http://127.28.218.193:45739/ using document root <none> and password file <none>
I20260806 22:52:13.526343 10679 fs_manager.cc:359] Metadata directory not provided
I20260806 22:52:13.526437 10679 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 22:52:13 dist-test-slave-1xrh krb5kdc[9589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056733, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.193@KRBTEST.COM for HTTP/127.28.218.212@KRBTEST.COM
I20260806 22:52:13.549894 10679 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:52:13.551242 10736 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:52:13.551455 10679 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 22:52:13.551591 10679 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/ts-0/wal
uuid: "3a45c988f3ee4979a63b75d595bea1b7"
format_stamp: "Formatted at 2026-08-06 22:52:12 on dist-test-slave-1xrh"
server_key: "f57f4600077051eb3c9089f7ebaf7a70"
server_key_iv: "a4c9fe01e18b730346a3ab342c41d1c9"
server_key_version: "kuduclusterkey@0"
I20260806 22:52:13.552105 10679 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:52:13.568666 10679 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:52:13.571067 10679 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:52:13.571312 10679 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:52:13.571632 10679 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:52:13.571998 10679 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:52:13.572048 10679 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:52:13.572077 10679 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:52:13.572103 10679 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:52:13.579653 10679 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.193:46773
I20260806 22:52:13.580145 10679 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/ts-0/data/info.pb
I20260806 22:52:13.580659 10857 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.193:46773 every 8 connection(s)
I20260806 22:52:13.588865 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 10679
I20260806 22:52:13.589167 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.28.218.212:53613/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.218.194:38293
--local_ip_for_outbound_sockets=127.28.218.194
--tserver_master_addrs=127.28.218.254:38697
--webserver_port=33809
--webserver_interface=127.28.218.194
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.194.keytab
--principal=kudu/127.28.218.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.218.212:41891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.28.218.212:53613/kms with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 22:52:13 dist-test-slave-1xrh krb5kdc[9589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056733, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.193@KRBTEST.COM for kudu/127.28.218.254@KRBTEST.COM
I20260806 22:52:13.596092 10860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:13.580931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:38697 (local address 127.28.218.193:48817)
0806 22:52:13.583239 (+ 2308us) negotiation.cc:107] Waiting for socket to connect
0806 22:52:13.583288 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 22:52:13.583954 (+ 666us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:52:13.584404 (+ 450us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:52:13.584409 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:52:13.584640 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:52:13.584913 (+ 273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:52:13.584919 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:52:13.586218 (+ 1299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:52:13.586219 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:52:13.587501 (+ 1282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:52:13.587505 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:52:13.588096 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:52:13.588097 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:52:13.588229 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:52:13.588707 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:52:13.588717 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:52:13.591772 (+ 3055us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:52:13.594049 (+ 2277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:52:13.594056 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:52:13.594058 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:52:13.594333 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:52:13.594656 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:52:13.594657 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:52:13.594657 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:52:13.594793 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:52:13.595348 (+ 555us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:52:13.595355 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:52:13.595514 (+ 159us) client_negotiation.cc:769] Sending connection context
0806 22:52:13.595611 (+ 97us) client_negotiation.cc:240] Negotiation successful
0806 22:52:13.595781 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2262,"thread_start_us":83,"threads_started":1}
I20260806 22:52:13.597203 10861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:13.581002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.193:48817 (local address 127.28.218.254:38697)
0806 22:52:13.581214 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0806 22:52:13.581221 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:52:13.583415 (+ 2194us) server_negotiation.cc:407] Connection header received
0806 22:52:13.584160 (+ 745us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:52:13.584167 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:52:13.584209 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:52:13.584320 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:52:13.585042 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:13.586107 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:13.587583 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:13.587973 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:13.588069 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:52:13.591892 (+ 3823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:52:13.591896 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:52:13.591896 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:52:13.591906 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:52:13.593946 (+ 2040us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:52:13.594446 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:52:13.594449 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:52:13.594449 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:52:13.594559 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:52:13.594890 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:52:13.594891 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:52:13.594892 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:52:13.595250 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:52:13.596688 (+ 1438us) server_negotiation.cc:1035] Waiting for connection context
0806 22:52:13.596780 (+ 92us) server_negotiation.cc:299] Negotiation successful
0806 22:52:13.596945 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":86,"threads_started":1}
I20260806 22:52:13.597911 10858 heartbeater.cc:347] Connected to a master server at 127.28.218.254:38697
I20260806 22:52:13.598035 10858 heartbeater.cc:464] Registering TS with master...
I20260806 22:52:13.598258 10858 heartbeater.cc:511] Master 127.28.218.254:38697 requested a full tablet report, sending...
W20260806 22:52:13.599253 10858 heartbeater.cc:644] Failed to heartbeat to 127.28.218.254:38697 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 22:52:13.599328 10858 heartbeater.cc:464] Registering TS with master...
I20260806 22:52:13.599400 10858 heartbeater.cc:511] Master 127.28.218.254:38697 requested a full tablet report, sending...
I20260806 22:52:13.599874 10858 heartbeater.cc:464] Registering TS with master...
I20260806 22:52:13.599956 10858 heartbeater.cc:511] Master 127.28.218.254:38697 requested a full tablet report, sending...
W20260806 22:52:13.600387 10858 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 22:52:13.691594 10862 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:52:13.691869 10862 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:52:13.691902 10862 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:52:13.693568 10862 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:52:13.693624 10862 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:52:13.693663 10862 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.218.194
I20260806 22:52:13.695873 10862 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:41891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.28.218.212:53613/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.194.keytab
--principal=kudu/127.28.218.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.218.194:38293
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.218.194
--webserver_port=33809
--tserver_master_addrs=127.28.218.254:38697
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10862
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.218.194
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:52:13.696259 10862 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:52:13.696568 10862 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:52:13.702680 10870 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:52:13.703385 10862 server_base.cc:1034] running on GCE node
W20260806 22:52:13.703387 10868 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:52:13.703662 10867 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:52:13.704006 10862 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:52:13.704253 10862 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:52:13.705394 10862 hybrid_clock.cc:648] HybridClock initialized: now 1786056733705350 us; error 56 us; skew 500 ppm
Aug 06 22:52:13 dist-test-slave-1xrh krb5kdc[9589](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056733, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:52:13.708321 10862 init.cc:377] Logged in from keytab as kudu/127.28.218.194@KRBTEST.COM (short username kudu)
I20260806 22:52:13.708796 10862 webserver.cc:460] Webserver started at http://127.28.218.194:33809/ using document root <none> and password file <none>
I20260806 22:52:13.709044 10862 fs_manager.cc:359] Metadata directory not provided
I20260806 22:52:13.709121 10862 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 22:52:13 dist-test-slave-1xrh krb5kdc[9589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056733, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.194@KRBTEST.COM for HTTP/127.28.218.212@KRBTEST.COM
I20260806 22:52:13.733546 10862 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.001s
I20260806 22:52:13.734647 10878 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:52:13.734826 10862 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 22:52:13.734917 10862 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/ts-1/wal
uuid: "b4f6bf590eb347008b1d9ca0ed128df8"
format_stamp: "Formatted at 2026-08-06 22:52:12 on dist-test-slave-1xrh"
server_key: "e940eaf14b1fd9d993615a66117863d3"
server_key_iv: "c0084e7787eb780a16d5aa5f8f8f3ef2"
server_key_version: "kuduclusterkey@0"
I20260806 22:52:13.735415 10862 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:52:13.756901 10862 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:52:13.759449 10862 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:52:13.759604 10862 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:52:13.759820 10862 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:52:13.760221 10862 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:52:13.760313 10862 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:52:13.760394 10862 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:52:13.760452 10862 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:52:13.776616 10862 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.194:38293
I20260806 22:52:13.777186 10862 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/ts-1/data/info.pb
I20260806 22:52:13.783308 11000 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.194:38293 every 8 connection(s)
I20260806 22:52:13.784210 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 10862
I20260806 22:52:13.784524 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.28.218.212:53613/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.218.195:33451
--local_ip_for_outbound_sockets=127.28.218.195
--tserver_master_addrs=127.28.218.254:38697
--webserver_port=35391
--webserver_interface=127.28.218.195
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.195.keytab
--principal=kudu/127.28.218.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.218.212:41891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.28.218.212:53613/kms with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 22:52:13 dist-test-slave-1xrh krb5kdc[9589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056733, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.194@KRBTEST.COM for kudu/127.28.218.254@KRBTEST.COM
I20260806 22:52:13.803639 10861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:13.779844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.194:41717 (local address 127.28.218.254:38697)
0806 22:52:13.779913 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0806 22:52:13.779915 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:52:13.786751 (+ 6836us) server_negotiation.cc:407] Connection header received
0806 22:52:13.787528 (+ 777us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:52:13.787529 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:52:13.787546 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:52:13.787631 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:52:13.788373 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:13.789177 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:13.791224 (+ 2047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:13.791524 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:13.791576 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:52:13.796225 (+ 4649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:52:13.796226 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:52:13.796226 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:52:13.796234 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:52:13.798114 (+ 1880us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:52:13.798911 (+ 797us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:52:13.798912 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:52:13.798913 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:52:13.798977 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:52:13.799314 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:52:13.799315 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:52:13.799315 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:52:13.799456 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:52:13.799521 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 22:52:13.803442 (+ 3921us) server_negotiation.cc:299] Negotiation successful
0806 22:52:13.803539 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 22:52:13.806429 11003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:13.779833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:38697 (local address 127.28.218.194:41717)
0806 22:52:13.786593 (+ 6760us) negotiation.cc:107] Waiting for socket to connect
0806 22:52:13.786645 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 22:52:13.787357 (+ 712us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:52:13.787718 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:52:13.787722 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:52:13.787963 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:52:13.788237 (+ 274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:52:13.788244 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:52:13.789294 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:52:13.789296 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:52:13.791084 (+ 1788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:52:13.791087 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:52:13.792614 (+ 1527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:52:13.792615 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:52:13.792745 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:52:13.793430 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:52:13.793441 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:52:13.796102 (+ 2661us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:52:13.798217 (+ 2115us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:52:13.798223 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:52:13.798225 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:52:13.798527 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:52:13.799059 (+ 532us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:52:13.799060 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:52:13.799061 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:52:13.799226 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:52:13.803061 (+ 3835us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:52:13.803068 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:52:13.803257 (+ 189us) client_negotiation.cc:769] Sending connection context
0806 22:52:13.805999 (+ 2742us) client_negotiation.cc:240] Negotiation successful
0806 22:52:13.806175 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6710,"thread_start_us":86,"threads_started":1}
I20260806 22:52:13.807241 11001 heartbeater.cc:347] Connected to a master server at 127.28.218.254:38697
I20260806 22:52:13.807340 11001 heartbeater.cc:464] Registering TS with master...
I20260806 22:52:13.807547 11001 heartbeater.cc:511] Master 127.28.218.254:38697 requested a full tablet report, sending...
W20260806 22:52:13.808243 11001 heartbeater.cc:644] Failed to heartbeat to 127.28.218.254:38697 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 22:52:13.808312 11001 heartbeater.cc:464] Registering TS with master...
I20260806 22:52:13.808369 11001 heartbeater.cc:511] Master 127.28.218.254:38697 requested a full tablet report, sending...
I20260806 22:52:13.808745 11001 heartbeater.cc:464] Registering TS with master...
I20260806 22:52:13.808818 11001 heartbeater.cc:511] Master 127.28.218.254:38697 requested a full tablet report, sending...
W20260806 22:52:13.809170 11001 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 22:52:13.888356 11004 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:52:13.888558 11004 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:52:13.888597 11004 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:52:13.890043 11004 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:52:13.890132 11004 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:52:13.890184 11004 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.218.195
I20260806 22:52:13.891949 11004 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:41891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.28.218.212:53613/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.195.keytab
--principal=kudu/127.28.218.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.218.195:33451
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.218.195
--webserver_port=35391
--tserver_master_addrs=127.28.218.254:38697
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11004
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.218.195
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:52:13.892175 11004 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:52:13.892395 11004 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:52:13.897208 11012 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:52:13.897209 11010 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:52:13.897756 11009 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:52:13.897971 11004 server_base.cc:1034] running on GCE node
I20260806 22:52:13.898151 11004 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:52:13.898353 11004 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:52:13.899483 11004 hybrid_clock.cc:648] HybridClock initialized: now 1786056733899460 us; error 42 us; skew 500 ppm
Aug 06 22:52:13 dist-test-slave-1xrh krb5kdc[9589](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056733, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:52:13.902089 11004 init.cc:377] Logged in from keytab as kudu/127.28.218.195@KRBTEST.COM (short username kudu)
I20260806 22:52:13.902616 11004 webserver.cc:460] Webserver started at http://127.28.218.195:35391/ using document root <none> and password file <none>
I20260806 22:52:13.902850 11004 fs_manager.cc:359] Metadata directory not provided
I20260806 22:52:13.902928 11004 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 22:52:13 dist-test-slave-1xrh krb5kdc[9589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056733, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.195@KRBTEST.COM for HTTP/127.28.218.212@KRBTEST.COM
I20260806 22:52:13.931537 11004 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20260806 22:52:13.935207 11021 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:52:13.935431 11004 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.000s
I20260806 22:52:13.935524 11004 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/ts-2/wal
uuid: "1fde50869d5b47018effa45a1417d1d1"
format_stamp: "Formatted at 2026-08-06 22:52:13 on dist-test-slave-1xrh"
server_key: "1839b6a86b3685cb7fd550f56b88a018"
server_key_iv: "8df9332bff6dfdb1f374f9d97babc2c2"
server_key_version: "kuduclusterkey@0"
I20260806 22:52:13.935981 11004 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:52:13.950815 11004 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:52:13.953060 11004 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:52:13.953246 11004 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:52:13.953516 11004 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:52:13.953895 11004 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:52:13.953985 11004 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:52:13.954053 11004 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:52:13.954108 11004 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:52:13.961226 11004 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.195:33451
I20260806 22:52:13.961750 11004 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/ts-2/data/info.pb
I20260806 22:52:13.962543 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 11004
I20260806 22:52:13.967401 11145 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.195:33451 every 8 connection(s)
Aug 06 22:52:13 dist-test-slave-1xrh krb5kdc[9589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056733, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.195@KRBTEST.COM for kudu/127.28.218.254@KRBTEST.COM
I20260806 22:52:13.974359 10861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:13.962837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.195:47635 (local address 127.28.218.254:38697)
0806 22:52:13.962889 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0806 22:52:13.962892 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:52:13.963287 (+ 395us) server_negotiation.cc:407] Connection header received
0806 22:52:13.964018 (+ 731us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:52:13.964019 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:52:13.964032 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:52:13.964064 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:52:13.964990 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:13.965783 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:13.967324 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:13.967642 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:13.967692 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:52:13.970074 (+ 2382us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:52:13.970076 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:52:13.970076 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:52:13.970083 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:52:13.971782 (+ 1699us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:52:13.972998 (+ 1216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:52:13.972999 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:52:13.972999 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:52:13.973059 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:52:13.973361 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:52:13.973361 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:52:13.973362 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:52:13.973496 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:52:13.973545 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 22:52:13.974173 (+ 628us) server_negotiation.cc:299] Negotiation successful
0806 22:52:13.974253 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 22:52:13.975064 11148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:13.962559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:38697 (local address 127.28.218.195:47635)
0806 22:52:13.963127 (+ 568us) negotiation.cc:107] Waiting for socket to connect
0806 22:52:13.963204 (+ 77us) client_negotiation.cc:174] Beginning negotiation
0806 22:52:13.963886 (+ 682us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:52:13.964281 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:52:13.964285 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:52:13.964569 (+ 284us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:52:13.964859 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:52:13.964866 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:52:13.965957 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:52:13.965958 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:52:13.967204 (+ 1246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:52:13.967207 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:52:13.967809 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:52:13.967810 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:52:13.967943 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:52:13.968522 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:52:13.968532 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:52:13.969967 (+ 1435us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:52:13.972613 (+ 2646us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:52:13.972619 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:52:13.972621 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:52:13.972903 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:52:13.973131 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:52:13.973132 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:52:13.973133 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:52:13.973258 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:52:13.973596 (+ 338us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:52:13.973602 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:52:13.973753 (+ 151us) client_negotiation.cc:769] Sending connection context
0806 22:52:13.974690 (+ 937us) client_negotiation.cc:240] Negotiation successful
0806 22:52:13.974848 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":511,"thread_start_us":71,"threads_started":1}
I20260806 22:52:13.975713 11146 heartbeater.cc:347] Connected to a master server at 127.28.218.254:38697
I20260806 22:52:13.975801 11146 heartbeater.cc:464] Registering TS with master...
I20260806 22:52:13.975982 11146 heartbeater.cc:511] Master 127.28.218.254:38697 requested a full tablet report, sending...
W20260806 22:52:13.976610 11146 heartbeater.cc:644] Failed to heartbeat to 127.28.218.254:38697 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 22:52:13.976676 11146 heartbeater.cc:464] Registering TS with master...
I20260806 22:52:13.976730 11146 heartbeater.cc:511] Master 127.28.218.254:38697 requested a full tablet report, sending...
I20260806 22:52:13.977051 11146 heartbeater.cc:464] Registering TS with master...
I20260806 22:52:13.977162 11146 heartbeater.cc:511] Master 127.28.218.254:38697 requested a full tablet report, sending...
W20260806 22:52:13.977478 11146 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 22:52:13.985761 11149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:13.962937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56552 (local address 127.28.218.254:38697)
0806 22:52:13.963082 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0806 22:52:13.963084 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:52:13.967737 (+ 4653us) server_negotiation.cc:407] Connection header received
0806 22:52:13.967764 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:52:13.967765 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:52:13.967777 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:52:13.967836 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:52:13.969077 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:13.969803 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:13.972600 (+ 2797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:13.972894 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:13.972965 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:52:13.982365 (+ 9400us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:52:13.982366 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:52:13.982367 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:52:13.982375 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:52:13.984174 (+ 1799us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:52:13.984622 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:52:13.984623 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:52:13.984623 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:52:13.984693 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:52:13.984924 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:52:13.984924 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:52:13.984925 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:52:13.985067 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:52:13.985236 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0806 22:52:13.985557 (+ 321us) server_negotiation.cc:299] Negotiation successful
0806 22:52:13.985662 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":72,"threads_started":1}
Aug 06 22:52:14 dist-test-slave-1xrh krb5kdc[9589](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786056734, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:52:14.600546 10858 heartbeater.cc:464] Registering TS with master...
I20260806 22:52:14.600708 10858 heartbeater.cc:511] Master 127.28.218.254:38697 requested a full tablet report, sending...
I20260806 22:52:14.601071 10637 ts_manager.cc:194] Registered new tserver with Master: 3a45c988f3ee4979a63b75d595bea1b7 (127.28.218.193:46773)
I20260806 22:52:14.602048 10637 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.218.193@KRBTEST.COM'} at 127.28.218.193:48817
I20260806 22:52:14.809361 11001 heartbeater.cc:464] Registering TS with master...
I20260806 22:52:14.809547 11001 heartbeater.cc:511] Master 127.28.218.254:38697 requested a full tablet report, sending...
I20260806 22:52:14.809845 10637 ts_manager.cc:194] Registered new tserver with Master: b4f6bf590eb347008b1d9ca0ed128df8 (127.28.218.194:38293)
I20260806 22:52:14.810402 10637 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.218.194@KRBTEST.COM'} at 127.28.218.194:41717
Aug 06 22:52:14 dist-test-slave-1xrh krb5kdc[9589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056734, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.254@KRBTEST.COM for HTTP/127.28.218.212@KRBTEST.COM
I20260806 22:52:14.977674 11146 heartbeater.cc:464] Registering TS with master...
I20260806 22:52:14.977831 11146 heartbeater.cc:511] Master 127.28.218.254:38697 requested a full tablet report, sending...
I20260806 22:52:14.978096 10637 ts_manager.cc:194] Registered new tserver with Master: 1fde50869d5b47018effa45a1417d1d1 (127.28.218.195:33451)
I20260806 22:52:14.978595 10637 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.218.195@KRBTEST.COM'} at 127.28.218.195:47635
I20260806 22:52:14.988641 29547 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:52:14.997483 11170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:14.989691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56564 (local address 127.28.218.254:38697)
0806 22:52:14.989822 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0806 22:52:14.989824 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:52:14.989834 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:52:14.989884 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:52:14.989884 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:52:14.989893 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:52:14.989953 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:52:14.990614 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:14.991082 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:14.992392 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:14.992583 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:14.992636 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:52:14.994520 (+ 1884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:52:14.994521 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:52:14.994521 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:52:14.994528 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:52:14.996059 (+ 1531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:52:14.996568 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:52:14.996569 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:52:14.996569 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:52:14.996620 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:52:14.996880 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:52:14.996881 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:52:14.996881 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:52:14.996981 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:52:14.997054 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 22:52:14.997299 (+ 245us) server_negotiation.cc:299] Negotiation successful
0806 22:52:14.997379 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":87,"threads_started":1}
I20260806 22:52:14.999717 10637 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56564:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:52:15.001430 10637 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:52:15.009881 11177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:15.004948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.194:38293 (local address 127.0.0.1:43394)
0806 22:52:15.005152 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0806 22:52:15.005163 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:52:15.005213 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:52:15.005754 (+ 541us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:52:15.005757 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:52:15.005759 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:52:15.005839 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:52:15.005841 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:52:15.006669 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:52:15.006670 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:52:15.007746 (+ 1076us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:52:15.007749 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:52:15.008729 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:52:15.008730 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:52:15.008806 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:52:15.009695 (+ 889us) client_negotiation.cc:769] Sending connection context
0806 22:52:15.009752 (+ 57us) client_negotiation.cc:240] Negotiation successful
0806 22:52:15.009803 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"mutex_wait_us":862,"thread_start_us":109,"threads_started":1}
I20260806 22:52:15.010515 11179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:15.005189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43394 (local address 127.28.218.194:38293)
0806 22:52:15.005541 (+ 352us) server_negotiation.cc:206] Beginning negotiation
0806 22:52:15.005546 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:52:15.005557 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:52:15.005592 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:52:15.005594 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:52:15.005610 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:52:15.005733 (+ 123us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:52:15.005924 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:15.006588 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:15.007844 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:15.008621 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:15.008830 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:52:15.008897 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 22:52:15.010377 (+ 1480us) server_negotiation.cc:299] Negotiation successful
0806 22:52:15.010438 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":300,"thread_start_us":65,"threads_started":1}
I20260806 22:52:15.011191 11175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:15.004828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.195:33451 (local address 127.0.0.1:41044)
0806 22:52:15.004993 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0806 22:52:15.005031 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 22:52:15.005089 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:52:15.007457 (+ 2368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:52:15.007458 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:52:15.007459 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:52:15.007513 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:52:15.007514 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:52:15.008717 (+ 1203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:52:15.008718 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:52:15.009614 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:52:15.009617 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:52:15.010914 (+ 1297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:52:15.010915 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:52:15.010992 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:52:15.010996 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:52:15.011051 (+ 55us) client_negotiation.cc:240] Negotiation successful
0806 22:52:15.011088 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":111,"threads_started":1}
I20260806 22:52:15.011238 11176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:15.004934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41044 (local address 127.28.218.195:33451)
0806 22:52:15.007232 (+ 2298us) server_negotiation.cc:206] Beginning negotiation
0806 22:52:15.007237 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:52:15.007248 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:52:15.007281 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:52:15.007283 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:52:15.007298 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:52:15.007375 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:52:15.007590 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:15.008612 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:15.010091 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:15.010804 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:15.010874 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:52:15.010931 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 22:52:15.011072 (+ 141us) server_negotiation.cc:299] Negotiation successful
0806 22:52:15.011111 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2267,"thread_start_us":64,"threads_started":1}
I20260806 22:52:15.012045 11178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:15.005092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.193:46773 (local address 127.0.0.1:36738)
0806 22:52:15.005295 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0806 22:52:15.005304 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:52:15.005352 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:52:15.007793 (+ 2441us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:52:15.007794 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:52:15.007795 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:52:15.007843 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:52:15.007845 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:52:15.009312 (+ 1467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:52:15.009312 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:52:15.010282 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:52:15.010285 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:52:15.011712 (+ 1427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:52:15.011713 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:52:15.011861 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:52:15.011866 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:52:15.011945 (+ 79us) client_negotiation.cc:240] Negotiation successful
0806 22:52:15.011983 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":96,"threads_started":1}
I20260806 22:52:15.012131 11180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:15.005320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36738 (local address 127.28.218.193:46773)
0806 22:52:15.007636 (+ 2316us) server_negotiation.cc:206] Beginning negotiation
0806 22:52:15.007640 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:52:15.007649 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:52:15.007679 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:52:15.007681 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:52:15.007694 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:52:15.008093 (+ 399us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:52:15.008179 (+ 86us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:15.009223 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:15.010781 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:15.011578 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:15.011712 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:52:15.011765 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 22:52:15.011972 (+ 207us) server_negotiation.cc:299] Negotiation successful
0806 22:52:15.012015 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2282,"thread_start_us":63,"threads_started":1}
I20260806 22:52:15.012320 11070 tablet_service.cc:1467] Processing CreateTablet for tablet 28d051ab78434dfc9b0d53a1de661d2d (DEFAULT_TABLE table=test-table [id=2504348764de4e9991de66e7d1d11dae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:52:15.012697 11070 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 28d051ab78434dfc9b0d53a1de661d2d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:52:15.015244 11181 tablet_bootstrap.cc:492] T 28d051ab78434dfc9b0d53a1de661d2d P 1fde50869d5b47018effa45a1417d1d1: Bootstrap starting.
I20260806 22:52:15.015600 10928 tablet_service.cc:1467] Processing CreateTablet for tablet 28d051ab78434dfc9b0d53a1de661d2d (DEFAULT_TABLE table=test-table [id=2504348764de4e9991de66e7d1d11dae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:52:15.015961 10928 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 28d051ab78434dfc9b0d53a1de661d2d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:52:15.016610 11181 tablet_bootstrap.cc:654] T 28d051ab78434dfc9b0d53a1de661d2d P 1fde50869d5b47018effa45a1417d1d1: Neither blocks nor log segments found. Creating new log.
I20260806 22:52:15.017232 11181 log.cc:824] T 28d051ab78434dfc9b0d53a1de661d2d P 1fde50869d5b47018effa45a1417d1d1: Log is configured to *not* fsync() on all Append() calls
I20260806 22:52:15.017889 11181 tablet_bootstrap.cc:492] T 28d051ab78434dfc9b0d53a1de661d2d P 1fde50869d5b47018effa45a1417d1d1: No bootstrap required, opened a new log
I20260806 22:52:15.017939 11181 ts_tablet_manager.cc:1397] T 28d051ab78434dfc9b0d53a1de661d2d P 1fde50869d5b47018effa45a1417d1d1: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 22:52:15.020344 11183 tablet_bootstrap.cc:492] T 28d051ab78434dfc9b0d53a1de661d2d P b4f6bf590eb347008b1d9ca0ed128df8: Bootstrap starting.
I20260806 22:52:15.020691 10789 tablet_service.cc:1467] Processing CreateTablet for tablet 28d051ab78434dfc9b0d53a1de661d2d (DEFAULT_TABLE table=test-table [id=2504348764de4e9991de66e7d1d11dae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:52:15.020828 11181 raft_consensus.cc:348] T 28d051ab78434dfc9b0d53a1de661d2d P 1fde50869d5b47018effa45a1417d1d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fde50869d5b47018effa45a1417d1d1" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 33451 } } peers { permanent_uuid: "b4f6bf590eb347008b1d9ca0ed128df8" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 38293 } } peers { permanent_uuid: "3a45c988f3ee4979a63b75d595bea1b7" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 46773 } }
I20260806 22:52:15.020998 11183 tablet_bootstrap.cc:654] T 28d051ab78434dfc9b0d53a1de661d2d P b4f6bf590eb347008b1d9ca0ed128df8: Neither blocks nor log segments found. Creating new log.
I20260806 22:52:15.020990 11181 raft_consensus.cc:374] T 28d051ab78434dfc9b0d53a1de661d2d P 1fde50869d5b47018effa45a1417d1d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:52:15.021028 11181 raft_consensus.cc:729] T 28d051ab78434dfc9b0d53a1de661d2d P 1fde50869d5b47018effa45a1417d1d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fde50869d5b47018effa45a1417d1d1, State: Initialized, Role: FOLLOWER
I20260806 22:52:15.021054 10789 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 28d051ab78434dfc9b0d53a1de661d2d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:52:15.021170 11181 consensus_queue.cc:260] T 28d051ab78434dfc9b0d53a1de661d2d P 1fde50869d5b47018effa45a1417d1d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fde50869d5b47018effa45a1417d1d1" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 33451 } } peers { permanent_uuid: "b4f6bf590eb347008b1d9ca0ed128df8" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 38293 } } peers { permanent_uuid: "3a45c988f3ee4979a63b75d595bea1b7" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 46773 } }
I20260806 22:52:15.021288 11183 log.cc:824] T 28d051ab78434dfc9b0d53a1de661d2d P b4f6bf590eb347008b1d9ca0ed128df8: Log is configured to *not* fsync() on all Append() calls
I20260806 22:52:15.021478 11181 ts_tablet_manager.cc:1428] T 28d051ab78434dfc9b0d53a1de661d2d P 1fde50869d5b47018effa45a1417d1d1: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260806 22:52:15.021603 11146 heartbeater.cc:503] Master 127.28.218.254:38697 was elected leader, sending a full tablet report...
I20260806 22:52:15.022526 11183 tablet_bootstrap.cc:492] T 28d051ab78434dfc9b0d53a1de661d2d P b4f6bf590eb347008b1d9ca0ed128df8: No bootstrap required, opened a new log
I20260806 22:52:15.022581 11183 ts_tablet_manager.cc:1397] T 28d051ab78434dfc9b0d53a1de661d2d P b4f6bf590eb347008b1d9ca0ed128df8: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:52:15.025099 11183 raft_consensus.cc:348] T 28d051ab78434dfc9b0d53a1de661d2d P b4f6bf590eb347008b1d9ca0ed128df8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fde50869d5b47018effa45a1417d1d1" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 33451 } } peers { permanent_uuid: "b4f6bf590eb347008b1d9ca0ed128df8" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 38293 } } peers { permanent_uuid: "3a45c988f3ee4979a63b75d595bea1b7" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 46773 } }
I20260806 22:52:15.025233 11183 raft_consensus.cc:374] T 28d051ab78434dfc9b0d53a1de661d2d P b4f6bf590eb347008b1d9ca0ed128df8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:52:15.025267 11183 raft_consensus.cc:729] T 28d051ab78434dfc9b0d53a1de661d2d P b4f6bf590eb347008b1d9ca0ed128df8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4f6bf590eb347008b1d9ca0ed128df8, State: Initialized, Role: FOLLOWER
I20260806 22:52:15.025341 11186 tablet_bootstrap.cc:492] T 28d051ab78434dfc9b0d53a1de661d2d P 3a45c988f3ee4979a63b75d595bea1b7: Bootstrap starting.
I20260806 22:52:15.025410 11183 consensus_queue.cc:260] T 28d051ab78434dfc9b0d53a1de661d2d P b4f6bf590eb347008b1d9ca0ed128df8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fde50869d5b47018effa45a1417d1d1" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 33451 } } peers { permanent_uuid: "b4f6bf590eb347008b1d9ca0ed128df8" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 38293 } } peers { permanent_uuid: "3a45c988f3ee4979a63b75d595bea1b7" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 46773 } }
I20260806 22:52:15.025699 11183 ts_tablet_manager.cc:1428] T 28d051ab78434dfc9b0d53a1de661d2d P b4f6bf590eb347008b1d9ca0ed128df8: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 22:52:15.025913 11001 heartbeater.cc:503] Master 127.28.218.254:38697 was elected leader, sending a full tablet report...
I20260806 22:52:15.026224 11186 tablet_bootstrap.cc:654] T 28d051ab78434dfc9b0d53a1de661d2d P 3a45c988f3ee4979a63b75d595bea1b7: Neither blocks nor log segments found. Creating new log.
I20260806 22:52:15.026608 11186 log.cc:824] T 28d051ab78434dfc9b0d53a1de661d2d P 3a45c988f3ee4979a63b75d595bea1b7: Log is configured to *not* fsync() on all Append() calls
I20260806 22:52:15.027465 11186 tablet_bootstrap.cc:492] T 28d051ab78434dfc9b0d53a1de661d2d P 3a45c988f3ee4979a63b75d595bea1b7: No bootstrap required, opened a new log
I20260806 22:52:15.027544 11186 ts_tablet_manager.cc:1397] T 28d051ab78434dfc9b0d53a1de661d2d P 3a45c988f3ee4979a63b75d595bea1b7: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
W20260806 22:52:15.030082 11002 tablet.cc:2369] T 28d051ab78434dfc9b0d53a1de661d2d P b4f6bf590eb347008b1d9ca0ed128df8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:52:15.030295 11186 raft_consensus.cc:348] T 28d051ab78434dfc9b0d53a1de661d2d P 3a45c988f3ee4979a63b75d595bea1b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fde50869d5b47018effa45a1417d1d1" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 33451 } } peers { permanent_uuid: "b4f6bf590eb347008b1d9ca0ed128df8" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 38293 } } peers { permanent_uuid: "3a45c988f3ee4979a63b75d595bea1b7" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 46773 } }
I20260806 22:52:15.030460 11186 raft_consensus.cc:374] T 28d051ab78434dfc9b0d53a1de661d2d P 3a45c988f3ee4979a63b75d595bea1b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:52:15.030522 11186 raft_consensus.cc:729] T 28d051ab78434dfc9b0d53a1de661d2d P 3a45c988f3ee4979a63b75d595bea1b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a45c988f3ee4979a63b75d595bea1b7, State: Initialized, Role: FOLLOWER
I20260806 22:52:15.030664 11186 consensus_queue.cc:260] T 28d051ab78434dfc9b0d53a1de661d2d P 3a45c988f3ee4979a63b75d595bea1b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fde50869d5b47018effa45a1417d1d1" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 33451 } } peers { permanent_uuid: "b4f6bf590eb347008b1d9ca0ed128df8" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 38293 } } peers { permanent_uuid: "3a45c988f3ee4979a63b75d595bea1b7" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 46773 } }
I20260806 22:52:15.030993 11186 ts_tablet_manager.cc:1428] T 28d051ab78434dfc9b0d53a1de661d2d P 3a45c988f3ee4979a63b75d595bea1b7: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260806 22:52:15.031033 10858 heartbeater.cc:503] Master 127.28.218.254:38697 was elected leader, sending a full tablet report...
I20260806 22:52:15.078953 11187 raft_consensus.cc:482] T 28d051ab78434dfc9b0d53a1de661d2d P b4f6bf590eb347008b1d9ca0ed128df8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:52:15.079126 11187 raft_consensus.cc:504] T 28d051ab78434dfc9b0d53a1de661d2d P b4f6bf590eb347008b1d9ca0ed128df8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fde50869d5b47018effa45a1417d1d1" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 33451 } } peers { permanent_uuid: "b4f6bf590eb347008b1d9ca0ed128df8" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 38293 } } peers { permanent_uuid: "3a45c988f3ee4979a63b75d595bea1b7" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 46773 } }
I20260806 22:52:15.079747 11187 leader_election.cc:288] T 28d051ab78434dfc9b0d53a1de661d2d P b4f6bf590eb347008b1d9ca0ed128df8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1fde50869d5b47018effa45a1417d1d1 (127.28.218.195:33451), 3a45c988f3ee4979a63b75d595bea1b7 (127.28.218.193:46773)
W20260806 22:52:15.081689 10859 tablet.cc:2369] T 28d051ab78434dfc9b0d53a1de661d2d P 3a45c988f3ee4979a63b75d595bea1b7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:52:15.084615 11176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:15.080186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.194:38097 (local address 127.28.218.195:33451)
0806 22:52:15.080537 (+ 351us) server_negotiation.cc:206] Beginning negotiation
0806 22:52:15.080540 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:52:15.080550 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:52:15.080579 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:52:15.080580 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:52:15.080590 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:52:15.080639 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:52:15.081218 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:15.081984 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:15.083556 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:15.084166 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:15.084213 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:52:15.084221 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 22:52:15.084490 (+ 269us) server_negotiation.cc:299] Negotiation successful
0806 22:52:15.084533 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322}
I20260806 22:52:15.085070 11180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:15.080776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.194:53641 (local address 127.28.218.193:46773)
0806 22:52:15.080823 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 22:52:15.080826 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:52:15.081062 (+ 236us) server_negotiation.cc:407] Connection header received
0806 22:52:15.081195 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:52:15.081196 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:52:15.081206 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:52:15.081239 (+ 33us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:52:15.081486 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:15.082185 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:15.083919 (+ 1734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:15.084544 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:15.084584 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:52:15.084591 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0806 22:52:15.084964 (+ 373us) server_negotiation.cc:299] Negotiation successful
0806 22:52:15.085006 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260806 22:52:15.085095 11190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:15.079977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.195:33451 (local address 127.28.218.194:38097)
0806 22:52:15.080357 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0806 22:52:15.080372 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 22:52:15.080476 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:52:15.080857 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:52:15.080858 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:52:15.080860 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:52:15.081104 (+ 244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:52:15.081107 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:52:15.082075 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:52:15.082076 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:52:15.083456 (+ 1380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:52:15.083459 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:52:15.084327 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:52:15.084328 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:52:15.084394 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:52:15.084406 (+ 12us) client_negotiation.cc:769] Sending connection context
0806 22:52:15.084954 (+ 548us) client_negotiation.cc:240] Negotiation successful
0806 22:52:15.084998 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":313,"thread_start_us":119,"threads_started":1}
I20260806 22:52:15.085621 11192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:15.080774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.193:46773 (local address 127.28.218.194:53641)
0806 22:52:15.080972 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0806 22:52:15.080992 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 22:52:15.081097 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:52:15.081302 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:52:15.081303 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:52:15.081305 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:52:15.081401 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:52:15.081402 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:52:15.082585 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:52:15.082586 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:52:15.083819 (+ 1233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:52:15.083822 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:52:15.084648 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:52:15.084649 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:52:15.084865 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:52:15.084869 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:52:15.085430 (+ 561us) client_negotiation.cc:240] Negotiation successful
0806 22:52:15.085472 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":89,"threads_started":1}
I20260806 22:52:15.085554 11095 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28d051ab78434dfc9b0d53a1de661d2d" candidate_uuid: "b4f6bf590eb347008b1d9ca0ed128df8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fde50869d5b47018effa45a1417d1d1" is_pre_election: true
I20260806 22:52:15.085759 11095 raft_consensus.cc:2454] T 28d051ab78434dfc9b0d53a1de661d2d P 1fde50869d5b47018effa45a1417d1d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bf590eb347008b1d9ca0ed128df8 in term 0.
I20260806 22:52:15.086046 10880 leader_election.cc:302] T 28d051ab78434dfc9b0d53a1de661d2d P b4f6bf590eb347008b1d9ca0ed128df8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fde50869d5b47018effa45a1417d1d1, b4f6bf590eb347008b1d9ca0ed128df8; no voters:
I20260806 22:52:15.086190 11187 raft_consensus.cc:2781] T 28d051ab78434dfc9b0d53a1de661d2d P b4f6bf590eb347008b1d9ca0ed128df8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:52:15.086190 10810 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28d051ab78434dfc9b0d53a1de661d2d" candidate_uuid: "b4f6bf590eb347008b1d9ca0ed128df8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a45c988f3ee4979a63b75d595bea1b7" is_pre_election: true
I20260806 22:52:15.086256 11187 raft_consensus.cc:482] T 28d051ab78434dfc9b0d53a1de661d2d P b4f6bf590eb347008b1d9ca0ed128df8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:52:15.086298 11187 raft_consensus.cc:3037] T 28d051ab78434dfc9b0d53a1de661d2d P b4f6bf590eb347008b1d9ca0ed128df8 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:52:15.086354 10810 raft_consensus.cc:2454] T 28d051ab78434dfc9b0d53a1de661d2d P 3a45c988f3ee4979a63b75d595bea1b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bf590eb347008b1d9ca0ed128df8 in term 0.
I20260806 22:52:15.087415 11187 raft_consensus.cc:504] T 28d051ab78434dfc9b0d53a1de661d2d P b4f6bf590eb347008b1d9ca0ed128df8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fde50869d5b47018effa45a1417d1d1" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 33451 } } peers { permanent_uuid: "b4f6bf590eb347008b1d9ca0ed128df8" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 38293 } } peers { permanent_uuid: "3a45c988f3ee4979a63b75d595bea1b7" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 46773 } }
I20260806 22:52:15.087553 11187 leader_election.cc:288] T 28d051ab78434dfc9b0d53a1de661d2d P b4f6bf590eb347008b1d9ca0ed128df8 [CANDIDATE]: Term 1 election: Requested vote from peers 1fde50869d5b47018effa45a1417d1d1 (127.28.218.195:33451), 3a45c988f3ee4979a63b75d595bea1b7 (127.28.218.193:46773)
I20260806 22:52:15.087787 11095 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28d051ab78434dfc9b0d53a1de661d2d" candidate_uuid: "b4f6bf590eb347008b1d9ca0ed128df8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fde50869d5b47018effa45a1417d1d1"
I20260806 22:52:15.087816 10810 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28d051ab78434dfc9b0d53a1de661d2d" candidate_uuid: "b4f6bf590eb347008b1d9ca0ed128df8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a45c988f3ee4979a63b75d595bea1b7"
I20260806 22:52:15.087867 11095 raft_consensus.cc:3037] T 28d051ab78434dfc9b0d53a1de661d2d P 1fde50869d5b47018effa45a1417d1d1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:52:15.087882 10810 raft_consensus.cc:3037] T 28d051ab78434dfc9b0d53a1de661d2d P 3a45c988f3ee4979a63b75d595bea1b7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:52:15.088956 10810 raft_consensus.cc:2454] T 28d051ab78434dfc9b0d53a1de661d2d P 3a45c988f3ee4979a63b75d595bea1b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bf590eb347008b1d9ca0ed128df8 in term 1.
I20260806 22:52:15.089090 11095 raft_consensus.cc:2454] T 28d051ab78434dfc9b0d53a1de661d2d P 1fde50869d5b47018effa45a1417d1d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bf590eb347008b1d9ca0ed128df8 in term 1.
I20260806 22:52:15.089150 10880 leader_election.cc:302] T 28d051ab78434dfc9b0d53a1de661d2d P b4f6bf590eb347008b1d9ca0ed128df8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a45c988f3ee4979a63b75d595bea1b7, b4f6bf590eb347008b1d9ca0ed128df8; no voters:
I20260806 22:52:15.089319 11187 raft_consensus.cc:2781] T 28d051ab78434dfc9b0d53a1de661d2d P b4f6bf590eb347008b1d9ca0ed128df8 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:52:15.089498 11187 raft_consensus.cc:686] T 28d051ab78434dfc9b0d53a1de661d2d P b4f6bf590eb347008b1d9ca0ed128df8 [term 1 LEADER]: Becoming Leader. State: Replica: b4f6bf590eb347008b1d9ca0ed128df8, State: Running, Role: LEADER
I20260806 22:52:15.089589 11187 consensus_queue.cc:237] T 28d051ab78434dfc9b0d53a1de661d2d P b4f6bf590eb347008b1d9ca0ed128df8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fde50869d5b47018effa45a1417d1d1" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 33451 } } peers { permanent_uuid: "b4f6bf590eb347008b1d9ca0ed128df8" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 38293 } } peers { permanent_uuid: "3a45c988f3ee4979a63b75d595bea1b7" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 46773 } }
I20260806 22:52:15.090392 10637 catalog_manager.cc:5495] T 28d051ab78434dfc9b0d53a1de661d2d P b4f6bf590eb347008b1d9ca0ed128df8 reported cstate change: term changed from 0 to 1, leader changed from <none> to b4f6bf590eb347008b1d9ca0ed128df8 (127.28.218.194). New cstate: current_term: 1 leader_uuid: "b4f6bf590eb347008b1d9ca0ed128df8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fde50869d5b47018effa45a1417d1d1" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 33451 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bf590eb347008b1d9ca0ed128df8" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 38293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a45c988f3ee4979a63b75d595bea1b7" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 46773 } health_report { overall_health: UNKNOWN } } }
I20260806 22:52:15.105744 11179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:15.102823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43404 (local address 127.28.218.194:38293)
0806 22:52:15.102858 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0806 22:52:15.102860 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:52:15.102914 (+ 54us) server_negotiation.cc:407] Connection header received
0806 22:52:15.103017 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:52:15.103017 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:52:15.103030 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:52:15.103103 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:52:15.103269 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:15.103896 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:15.104882 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:15.105179 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:15.105251 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:52:15.105415 (+ 164us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:52:15.105520 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:52:15.105600 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 22:52:15.105641 (+ 41us) server_negotiation.cc:299] Negotiation successful
0806 22:52:15.105662 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 22:52:15.107936 10810 raft_consensus.cc:1261] T 28d051ab78434dfc9b0d53a1de661d2d P 3a45c988f3ee4979a63b75d595bea1b7 [term 1 FOLLOWER]: Refusing update from remote peer b4f6bf590eb347008b1d9ca0ed128df8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:52:15.107961 11095 raft_consensus.cc:1261] T 28d051ab78434dfc9b0d53a1de661d2d P 1fde50869d5b47018effa45a1417d1d1 [term 1 FOLLOWER]: Refusing update from remote peer b4f6bf590eb347008b1d9ca0ed128df8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:52:15.108351 11193 consensus_queue.cc:1035] T 28d051ab78434dfc9b0d53a1de661d2d P b4f6bf590eb347008b1d9ca0ed128df8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fde50869d5b47018effa45a1417d1d1" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 33451 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:52:15.108515 11187 consensus_queue.cc:1035] T 28d051ab78434dfc9b0d53a1de661d2d P b4f6bf590eb347008b1d9ca0ed128df8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a45c988f3ee4979a63b75d595bea1b7" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 46773 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:52:15.112171 11196 mvcc.cc:205] Tried to move back new op lower bound from 7315688386999484416 to 7315688386927394816. Current Snapshot: MvccSnapshot[applied={T|T < 7315688386999484416}]
I20260806 22:52:15.112643 11199 mvcc.cc:205] Tried to move back new op lower bound from 7315688386999484416 to 7315688386927394816. Current Snapshot: MvccSnapshot[applied={T|T < 7315688386999484416}]
I20260806 22:52:15.116344 11197 mvcc.cc:205] Tried to move back new op lower bound from 7315688386999484416 to 7315688386927394816. Current Snapshot: MvccSnapshot[applied={T|T < 7315688386999484416}]
I20260806 22:52:15.121565 10637 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56564:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:52:15.121661 10637 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56564:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:52:15.122448 10637 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 96cc599743cd4165823d9f6a6954420f: Sending DeleteTablet for 3 replicas of tablet 28d051ab78434dfc9b0d53a1de661d2d
I20260806 22:52:15.123035 11070 tablet_service.cc:1514] Processing DeleteTablet for tablet 28d051ab78434dfc9b0d53a1de661d2d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:52:15 UTC) from {username='kudu', principal='kudu/127.28.218.254@KRBTEST.COM'} at 127.0.0.1:41044
I20260806 22:52:15.123382 10789 tablet_service.cc:1514] Processing DeleteTablet for tablet 28d051ab78434dfc9b0d53a1de661d2d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:52:15 UTC) from {username='kudu', principal='kudu/127.28.218.254@KRBTEST.COM'} at 127.0.0.1:36738
I20260806 22:52:15.124033 10928 tablet_service.cc:1514] Processing DeleteTablet for tablet 28d051ab78434dfc9b0d53a1de661d2d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:52:15 UTC) from {username='kudu', principal='kudu/127.28.218.254@KRBTEST.COM'} at 127.0.0.1:43394
I20260806 22:52:15.124740 11208 tablet_replica.cc:330] T 28d051ab78434dfc9b0d53a1de661d2d P b4f6bf590eb347008b1d9ca0ed128df8: stopping tablet replica
I20260806 22:52:15.124893 11208 raft_consensus.cc:2229] T 28d051ab78434dfc9b0d53a1de661d2d P b4f6bf590eb347008b1d9ca0ed128df8 [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:52:15.125092 11208 raft_consensus.cc:2258] T 28d051ab78434dfc9b0d53a1de661d2d P b4f6bf590eb347008b1d9ca0ed128df8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:52:15.125449 11206 tablet_replica.cc:330] T 28d051ab78434dfc9b0d53a1de661d2d P 1fde50869d5b47018effa45a1417d1d1: stopping tablet replica
I20260806 22:52:15.125612 11206 raft_consensus.cc:2229] T 28d051ab78434dfc9b0d53a1de661d2d P 1fde50869d5b47018effa45a1417d1d1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:52:15.125782 11206 raft_consensus.cc:2258] T 28d051ab78434dfc9b0d53a1de661d2d P 1fde50869d5b47018effa45a1417d1d1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:52:15.128895 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 10679
I20260806 22:52:15.129072 11208 ts_tablet_manager.cc:1905] T 28d051ab78434dfc9b0d53a1de661d2d P b4f6bf590eb347008b1d9ca0ed128df8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:52:15.131726 11208 ts_tablet_manager.cc:1918] T 28d051ab78434dfc9b0d53a1de661d2d P b4f6bf590eb347008b1d9ca0ed128df8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:52:15.131897 11208 log.cc:1196] T 28d051ab78434dfc9b0d53a1de661d2d P b4f6bf590eb347008b1d9ca0ed128df8: Deleting WAL directory at /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/ts-1/wal/wals/28d051ab78434dfc9b0d53a1de661d2d
I20260806 22:52:15.132282 11208 ts_tablet_manager.cc:1939] T 28d051ab78434dfc9b0d53a1de661d2d P b4f6bf590eb347008b1d9ca0ed128df8: Deleting consensus metadata
I20260806 22:52:15.137544 10619 catalog_manager.cc:4841] TS b4f6bf590eb347008b1d9ca0ed128df8 (127.28.218.194:38293): tablet 28d051ab78434dfc9b0d53a1de661d2d (table test-table [id=2504348764de4e9991de66e7d1d11dae]) successfully deleted
W20260806 22:52:15.137871 10620 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.28.218.193:46773 (error 108)
I20260806 22:52:15.138617 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 10862
I20260806 22:52:15.140399 11178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:15.140255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.193:46773 (local address 127.0.0.1:36744)
0806 22:52:15.140297 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0806 22:52:15.140333 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.193:46773: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260806 22:52:15.140604 10620 catalog_manager.cc:4584] TS 3a45c988f3ee4979a63b75d595bea1b7 (127.28.218.193:46773): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 28d051ab78434dfc9b0d53a1de661d2d: Network error: Client connection negotiation failed: client connection to 127.28.218.193:46773: connect: Connection refused (error 111)
I20260806 22:52:15.140764 11206 ts_tablet_manager.cc:1905] T 28d051ab78434dfc9b0d53a1de661d2d P 1fde50869d5b47018effa45a1417d1d1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:52:15.142946 11206 ts_tablet_manager.cc:1918] T 28d051ab78434dfc9b0d53a1de661d2d P 1fde50869d5b47018effa45a1417d1d1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:52:15.143066 11206 log.cc:1196] T 28d051ab78434dfc9b0d53a1de661d2d P 1fde50869d5b47018effa45a1417d1d1: Deleting WAL directory at /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056586788339-29547-0/minicluster-data/ts-2/wal/wals/28d051ab78434dfc9b0d53a1de661d2d
I20260806 22:52:15.143481 11206 ts_tablet_manager.cc:1939] T 28d051ab78434dfc9b0d53a1de661d2d P 1fde50869d5b47018effa45a1417d1d1: Deleting consensus metadata
I20260806 22:52:15.143975 10620 catalog_manager.cc:4841] TS 1fde50869d5b47018effa45a1417d1d1 (127.28.218.195:33451): tablet 28d051ab78434dfc9b0d53a1de661d2d (table test-table [id=2504348764de4e9991de66e7d1d11dae]) successfully deleted
I20260806 22:52:15.145219 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 11004
I20260806 22:52:15.152315 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 10600
I20260806 22:52:15.161360 29547 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260806 22:52:15.505796 29547 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260806 22:52:15.505960 29547 mini_ranger.cc:66] Stopping Ranger...
I20260806 22:52:15.849761 29547 mini_ranger.cc:68] Stopped Ranger
2026-08-06 22:52:15.849 UTC [9620] LOG: received smart shutdown request
2026-08-06 22:52:15.854 UTC [9620] LOG: background worker "logical replication launcher" (PID 9629) exited with exit code 1
2026-08-06 22:52:15.854 UTC [9624] LOG: shutting down
2026-08-06 22:52:16.019 UTC [9620] LOG: database system is shut down
2026-08-06T22:52:16Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (47542 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056586788339-29547-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:52:16 dist-test-slave-1xrh krb5kdc[11219](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:52:16 dist-test-slave-1xrh krb5kdc[11219](info): set up 2 sockets
Aug 06 22:52:16 dist-test-slave-1xrh krb5kdc[11219](info): commencing operation
krb5kdc: starting...
W20260806 22:52:17.178997 29547 mini_kdc.cc:121] Time spent starting KDC: real 1.029s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056586788339-29547-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056586788339-29547-0/krb5kdc/test-user.keytab.
Aug 06 22:52:17 dist-test-slave-1xrh krb5kdc[11219](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056737, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:52:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:52:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.218.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.218.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.218.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab.
Entry for principal kudu/127.28.218.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab.
I20260806 22:52:17.298077 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056586788339-29547-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056586788339-29547-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056586788339-29547-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.218.254:44059
--webserver_interface=127.28.218.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab
--principal=kudu/127.28.218.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.218.212:36525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.218.254:44059,127.28.218.253:37573,127.28.218.252:36357
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:52:17.383556 11233 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:52:17.383755 11233 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:52:17.383777 11233 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:52:17.384931 11233 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:52:17.384974 11233 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:52:17.384989 11233 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:52:17.385002 11233 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:52:17.385015 11233 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:52:17.386495 11233 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:36525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056586788339-29547-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056586788339-29547-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.218.254:44059,127.28.218.253:37573,127.28.218.252:36357
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab
--principal=kudu/127.28.218.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.218.254:44059
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.218.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11233
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056586788339-29547-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:52:17.386785 11233 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:52:17.387008 11233 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:52:17.391959 11240 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:52:17.391933 11243 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:52:17.391927 11241 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:52:17.392000 11233 server_base.cc:1034] running on GCE node
I20260806 22:52:17.392442 11233 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:52:17.392765 11233 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:52:17.393934 11233 hybrid_clock.cc:648] HybridClock initialized: now 1786056737393915 us; error 38 us; skew 500 ppm
Aug 06 22:52:17 dist-test-slave-1xrh krb5kdc[11219](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056737, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:52:17.399631 11233 init.cc:377] Logged in from keytab as kudu/127.28.218.254@KRBTEST.COM (short username kudu)
I20260806 22:52:17.400067 11233 webserver.cc:460] Webserver started at http://127.28.218.254:41495/ using document root <none> and password file <none>
I20260806 22:52:17.400313 11233 fs_manager.cc:359] Metadata directory not provided
I20260806 22:52:17.400381 11233 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:52:17.400537 11233 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:52:17.401610 11233 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056586788339-29547-0/minicluster-data/master-0/data/instance:
uuid: "943296ed5a2741558301ed7f21723f05"
format_stamp: "Formatted at 2026-08-06 22:52:17 on dist-test-slave-1xrh"
server_key: "c9c74edaea8cc4e30a68c11a9e022745"
server_key_iv: "ca8bf4e49a8564d8d540680f506fbd43"
server_key_version: "encryptionkey@0"
I20260806 22:52:17.401978 11233 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056586788339-29547-0/minicluster-data/master-0/wal/instance:
uuid: "943296ed5a2741558301ed7f21723f05"
format_stamp: "Formatted at 2026-08-06 22:52:17 on dist-test-slave-1xrh"
server_key: "c9c74edaea8cc4e30a68c11a9e022745"
server_key_iv: "ca8bf4e49a8564d8d540680f506fbd43"
server_key_version: "encryptionkey@0"
I20260806 22:52:17.403371 11233 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 22:52:17.404047 11252 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:52:17.404217 11233 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:52:17.404304 11233 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056586788339-29547-0/minicluster-data/master-0/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056586788339-29547-0/minicluster-data/master-0/wal
uuid: "943296ed5a2741558301ed7f21723f05"
format_stamp: "Formatted at 2026-08-06 22:52:17 on dist-test-slave-1xrh"
server_key: "c9c74edaea8cc4e30a68c11a9e022745"
server_key_iv: "ca8bf4e49a8564d8d540680f506fbd43"
server_key_version: "encryptionkey@0"
I20260806 22:52:17.404394 11233 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056586788339-29547-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056586788339-29547-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056586788339-29547-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:52:17.423833 11233 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:52:17.425626 11233 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:52:17.425787 11233 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:52:17.429939 11233 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.254:44059
I20260806 22:52:17.429972 11307 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.254:44059 every 8 connection(s)
I20260806 22:52:17.430330 11233 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
I20260806 22:52:17.431212 11308 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:52:17.432544 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 11233
I20260806 22:52:17.432687 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056586788339-29547-0/minicluster-data/master-0/wal/instance
I20260806 22:52:17.432921 29547 external_mini_cluster.cc:1428] Setting key e3ed64f0c0a6eec92042eb30b4280d6f
I20260806 22:52:17.432843 11308 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.218.254" port: 44059 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 22:52:17 dist-test-slave-1xrh krb5kdc[11219](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056737, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.254@KRBTEST.COM for kudu/127.28.218.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.28.218.253@KRBTEST.COM; defaulting to no policy
I20260806 22:52:17.443908 11311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:17.433588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:44059 (local address 127.0.0.1:41132)
0806 22:52:17.433776 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0806 22:52:17.433815 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 22:52:17.434307 (+ 492us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:52:17.434574 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:52:17.434576 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:52:17.434852 (+ 276us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:52:17.434998 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:52:17.435002 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:52:17.436157 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:52:17.436158 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:52:17.437250 (+ 1092us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:52:17.437253 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:52:17.438004 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:52:17.438004 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:52:17.438056 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:52:17.438664 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:52:17.438673 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:52:17.440587 (+ 1914us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:52:17.442404 (+ 1817us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:52:17.442410 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:52:17.442412 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:52:17.442604 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:52:17.442850 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:52:17.442851 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:52:17.442851 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:52:17.442941 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:52:17.443334 (+ 393us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:52:17.443339 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:52:17.443396 (+ 57us) client_negotiation.cc:769] Sending connection context
0806 22:52:17.443499 (+ 103us) client_negotiation.cc:240] Negotiation successful
0806 22:52:17.443729 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"mutex_wait_us":99,"spinlock_wait_cycles":7936,"thread_start_us":112,"threads_started":1}
I20260806 22:52:17.443925 11310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:17.433601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41132 (local address 127.28.218.254:44059)
0806 22:52:17.433760 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0806 22:52:17.433769 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:52:17.433915 (+ 146us) server_negotiation.cc:407] Connection header received
0806 22:52:17.434458 (+ 543us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:52:17.434460 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:52:17.434476 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:52:17.434569 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:52:17.435096 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:17.436053 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:17.437343 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:17.437809 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:17.437915 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:52:17.440681 (+ 2766us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:52:17.440686 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:52:17.440687 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:52:17.440692 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:52:17.442319 (+ 1627us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:52:17.442678 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:52:17.442680 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:52:17.442681 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:52:17.442783 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:52:17.443004 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:52:17.443006 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:52:17.443006 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:52:17.443266 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:52:17.443355 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 22:52:17.443535 (+ 180us) server_negotiation.cc:299] Negotiation successful
0806 22:52:17.443666 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":83,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.218.253@KRBTEST.COM" created.
I20260806 22:52:17.444628 11308 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.218.253" port: 37573 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:52:17.444974 11311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:17.444844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.253:37573 (local address 127.0.0.1:45384)
0806 22:52:17.444891 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0806 22:52:17.444941 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.253:37573: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260806 22:52:17.445099 11254 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.28.218.253:37573: connect: Connection refused (error 111)
I20260806 22:52:17.445470 11311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:17.445387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.253:37573 (local address 127.0.0.1:45392)
0806 22:52:17.445426 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0806 22:52:17.445450 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.253:37573: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260806 22:52:17.445621 11308 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.218.253:37573: Network error: Client connection negotiation failed: client connection to 127.28.218.253:37573: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.218.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.253.keytab.
Entry for principal kudu/127.28.218.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.253.keytab.
I20260806 22:52:17.460011 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056586788339-29547-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056586788339-29547-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056586788339-29547-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056586788339-29547-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.218.253:37573
--webserver_interface=127.28.218.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.253.keytab
--principal=kudu/127.28.218.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.218.212:36525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.218.254:44059,127.28.218.253:37573,127.28.218.252:36357
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:52:17.494884 11308 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.218.253" port: 37573 } attempt: 1
I20260806 22:52:17.495378 11311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:17.495255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.253:37573 (local address 127.0.0.1:45408)
0806 22:52:17.495317 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0806 22:52:17.495347 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.253:37573: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260806 22:52:17.495754 11311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:17.495662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.253:37573 (local address 127.0.0.1:45420)
0806 22:52:17.495707 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0806 22:52:17.495730 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.253:37573: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260806 22:52:17.495836 11308 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.218.253:37573: Network error: Client connection negotiation failed: client connection to 127.28.218.253:37573: connect: Connection refused (error 111)
W20260806 22:52:17.542320 11315 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:52:17.542522 11315 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:52:17.542544 11315 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:52:17.543673 11315 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:52:17.543713 11315 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:52:17.543728 11315 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:52:17.543741 11315 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:52:17.543792 11315 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:52:17.545284 11315 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:36525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056586788339-29547-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056586788339-29547-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.28.218.254:44059,127.28.218.253:37573,127.28.218.252:36357
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.253.keytab
--principal=kudu/127.28.218.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.218.253:37573
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056586788339-29547-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.28.218.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11315
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056586788339-29547-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:52:17.545522 11315 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:52:17.545743 11315 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:52:17.550380 11321 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:52:17.550380 11323 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:52:17.550431 11320 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:52:17.550683 11315 server_base.cc:1034] running on GCE node
I20260806 22:52:17.551000 11315 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:52:17.551330 11315 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:52:17.552469 11315 hybrid_clock.cc:648] HybridClock initialized: now 1786056737552453 us; error 36 us; skew 500 ppm
Aug 06 22:52:17 dist-test-slave-1xrh krb5kdc[11219](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056737, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:52:17.554908 11315 init.cc:377] Logged in from keytab as kudu/127.28.218.253@KRBTEST.COM (short username kudu)
I20260806 22:52:17.555388 11315 webserver.cc:460] Webserver started at http://127.28.218.253:39163/ using document root <none> and password file <none>
I20260806 22:52:17.555634 11315 fs_manager.cc:359] Metadata directory not provided
I20260806 22:52:17.555701 11315 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:52:17.555837 11315 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:52:17.556815 11315 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056586788339-29547-0/minicluster-data/master-1/data/instance:
uuid: "fb3ec65dc40846beab22bb7218ab8f56"
format_stamp: "Formatted at 2026-08-06 22:52:17 on dist-test-slave-1xrh"
server_key: "3b6a576088aaa5edcb5ae8f81932d739"
server_key_iv: "c145456302c78281a54daf10a9b6890c"
server_key_version: "encryptionkey@0"
I20260806 22:52:17.557161 11315 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056586788339-29547-0/minicluster-data/master-1/wal/instance:
uuid: "fb3ec65dc40846beab22bb7218ab8f56"
format_stamp: "Formatted at 2026-08-06 22:52:17 on dist-test-slave-1xrh"
server_key: "3b6a576088aaa5edcb5ae8f81932d739"
server_key_iv: "c145456302c78281a54daf10a9b6890c"
server_key_version: "encryptionkey@0"
I20260806 22:52:17.558393 11315 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:52:17.559224 11330 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:52:17.559383 11315 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:52:17.559473 11315 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056586788339-29547-0/minicluster-data/master-1/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056586788339-29547-0/minicluster-data/master-1/wal
uuid: "fb3ec65dc40846beab22bb7218ab8f56"
format_stamp: "Formatted at 2026-08-06 22:52:17 on dist-test-slave-1xrh"
server_key: "3b6a576088aaa5edcb5ae8f81932d739"
server_key_iv: "c145456302c78281a54daf10a9b6890c"
server_key_version: "encryptionkey@0"
I20260806 22:52:17.559554 11315 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056586788339-29547-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056586788339-29547-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056586788339-29547-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:52:17.563970 11308 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.218.253" port: 37573 } attempt: 2
I20260806 22:52:17.564381 11311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:17.564256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.253:37573 (local address 127.0.0.1:45430)
0806 22:52:17.564318 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0806 22:52:17.564352 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.253:37573: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260806 22:52:17.564677 11311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:17.564585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.253:37573 (local address 127.0.0.1:45442)
0806 22:52:17.564626 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0806 22:52:17.564650 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.253:37573: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260806 22:52:17.564761 11308 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.218.253:37573: Network error: Client connection negotiation failed: client connection to 127.28.218.253:37573: connect: Connection refused (error 111)
I20260806 22:52:17.572597 11315 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:52:17.574219 11315 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:52:17.574366 11315 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:52:17.578408 11315 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.253:37573
I20260806 22:52:17.578442 11387 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.253:37573 every 8 connection(s)
I20260806 22:52:17.578799 11315 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056586788339-29547-0/minicluster-data/master-1/data/info.pb
I20260806 22:52:17.579694 11388 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:52:17.581219 11388 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.218.254" port: 44059 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:52:17.584538 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 11315
I20260806 22:52:17.584662 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056586788339-29547-0/minicluster-data/master-1/wal/instance
I20260806 22:52:17.584880 29547 external_mini_cluster.cc:1428] Setting key 11407d4aa2808fc7e170c2d23318fd13
Aug 06 22:52:17 dist-test-slave-1xrh krb5kdc[11219](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056737, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.253@KRBTEST.COM for kudu/127.28.218.254@KRBTEST.COM
I20260806 22:52:17.589846 11310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:17.581848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41140 (local address 127.28.218.254:44059)
0806 22:52:17.581917 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0806 22:52:17.581918 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:52:17.582104 (+ 186us) server_negotiation.cc:407] Connection header received
0806 22:52:17.582507 (+ 403us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:52:17.582509 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:52:17.582519 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:52:17.582581 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:52:17.583193 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:17.583690 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:17.584631 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:17.584883 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:17.584967 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:52:17.586996 (+ 2029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:52:17.586997 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:52:17.586998 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:52:17.587006 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:52:17.588437 (+ 1431us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:52:17.588882 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:52:17.588883 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:52:17.588884 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:52:17.588960 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:52:17.589222 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:52:17.589223 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:52:17.589223 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:52:17.589338 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:52:17.589407 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 22:52:17.589667 (+ 260us) server_negotiation.cc:299] Negotiation successful
0806 22:52:17.589755 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 22:52:17.590044 11390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:17.581811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:44059 (local address 127.0.0.1:41140)
0806 22:52:17.581955 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0806 22:52:17.581995 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 22:52:17.582427 (+ 432us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:52:17.582632 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:52:17.582635 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:52:17.582867 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:52:17.583053 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:52:17.583057 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:52:17.583766 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:52:17.583766 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:52:17.584537 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:52:17.584539 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:52:17.584978 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:52:17.584979 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:52:17.585103 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:52:17.585589 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:52:17.585597 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:52:17.586902 (+ 1305us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:52:17.588575 (+ 1673us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:52:17.588583 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:52:17.588585 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:52:17.588807 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:52:17.589035 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:52:17.589036 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:52:17.589036 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:52:17.589146 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:52:17.589399 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:52:17.589405 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:52:17.589539 (+ 134us) client_negotiation.cc:769] Sending connection context
0806 22:52:17.589646 (+ 107us) client_negotiation.cc:240] Negotiation successful
0806 22:52:17.589797 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":72,"threads_started":1}
I20260806 22:52:17.590493 11388 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.218.253" port: 37573 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.28.218.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.218.252@KRBTEST.COM" created.
Aug 06 22:52:17 dist-test-slave-1xrh krb5kdc[11219](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056737, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.253@KRBTEST.COM for kudu/127.28.218.253@KRBTEST.COM
I20260806 22:52:17.600134 11392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:17.590782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45456 (local address 127.28.218.253:37573)
0806 22:52:17.590932 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0806 22:52:17.590937 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:52:17.590963 (+ 26us) server_negotiation.cc:407] Connection header received
0806 22:52:17.590989 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:52:17.590991 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:52:17.591007 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:52:17.591095 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:52:17.591400 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:17.592555 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:17.593193 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:17.593492 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:17.593567 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:52:17.596932 (+ 3365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:52:17.596940 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:52:17.596941 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:52:17.596957 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:52:17.598822 (+ 1865us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:52:17.599281 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:52:17.599283 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:52:17.599283 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:52:17.599336 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:52:17.599562 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:52:17.599562 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:52:17.599563 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:52:17.599690 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:52:17.599746 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 22:52:17.599920 (+ 174us) server_negotiation.cc:299] Negotiation successful
0806 22:52:17.600038 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":46,"server-negotiator.queue_time_us":117,"thread_start_us":79,"threads_started":1}
I20260806 22:52:17.600137 11390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:17.590731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.253:37573 (local address 127.0.0.1:45456)
0806 22:52:17.590773 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0806 22:52:17.590783 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:52:17.590821 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:52:17.591108 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:52:17.591108 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:52:17.591261 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:52:17.591305 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:52:17.591306 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:52:17.592650 (+ 1344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:52:17.592651 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:52:17.593119 (+ 468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:52:17.593120 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:52:17.593565 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:52:17.593566 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:52:17.593631 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:52:17.594005 (+ 374us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:52:17.594009 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:52:17.596837 (+ 2828us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:52:17.598945 (+ 2108us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:52:17.598946 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:52:17.598946 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:52:17.599205 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:52:17.599402 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:52:17.599403 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:52:17.599404 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:52:17.599476 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:52:17.599750 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:52:17.599751 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:52:17.599799 (+ 48us) client_negotiation.cc:769] Sending connection context
0806 22:52:17.599876 (+ 77us) client_negotiation.cc:240] Negotiation successful
0806 22:52:17.599980 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260806 22:52:17.600853 11388 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.218.252" port: 36357 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:52:17.601248 11390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:17.601077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.252:36357 (local address 127.0.0.1:50824)
0806 22:52:17.601149 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0806 22:52:17.601216 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.252:36357: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260806 22:52:17.601399 11334 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.28.218.252:36357: connect: Connection refused (error 111)
I20260806 22:52:17.601850 11390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:17.601739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.252:36357 (local address 127.0.0.1:50834)
0806 22:52:17.601785 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0806 22:52:17.601819 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.252:36357: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260806 22:52:17.601964 11388 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.218.252:36357: Network error: Client connection negotiation failed: client connection to 127.28.218.252:36357: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.218.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.252.keytab.
Entry for principal kudu/127.28.218.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.252.keytab.
I20260806 22:52:17.613480 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056586788339-29547-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056586788339-29547-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056586788339-29547-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056586788339-29547-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.218.252:36357
--webserver_interface=127.28.218.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.252.keytab
--principal=kudu/127.28.218.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.218.212:36525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.218.254:44059,127.28.218.253:37573,127.28.218.252:36357
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:52:17.651216 11388 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.218.252" port: 36357 } attempt: 1
I20260806 22:52:17.651717 11390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:17.651563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.252:36357 (local address 127.0.0.1:50844)
0806 22:52:17.651628 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0806 22:52:17.651678 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.252:36357: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260806 22:52:17.652096 11390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:17.652007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.252:36357 (local address 127.0.0.1:50848)
0806 22:52:17.652048 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0806 22:52:17.652074 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.252:36357: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260806 22:52:17.652199 11388 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.218.252:36357: Network error: Client connection negotiation failed: client connection to 127.28.218.252:36357: connect: Connection refused (error 111)
I20260806 22:52:17.655902 11308 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.218.253" port: 37573 } attempt: 3
Aug 06 22:52:17 dist-test-slave-1xrh krb5kdc[11219](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056737, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.254@KRBTEST.COM for kudu/127.28.218.253@KRBTEST.COM
I20260806 22:52:17.663165 11392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:17.656250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45470 (local address 127.28.218.253:37573)
0806 22:52:17.656301 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0806 22:52:17.656303 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:52:17.656341 (+ 38us) server_negotiation.cc:407] Connection header received
0806 22:52:17.656364 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:52:17.656365 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:52:17.656374 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:52:17.656414 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:52:17.656781 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:17.657343 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:17.658325 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:17.658528 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:17.658576 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:52:17.660603 (+ 2027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:52:17.660604 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:52:17.660605 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:52:17.660616 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:52:17.662029 (+ 1413us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:52:17.662345 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:52:17.662347 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:52:17.662347 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:52:17.662430 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:52:17.662683 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:52:17.662683 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:52:17.662684 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:52:17.662776 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:52:17.662823 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0806 22:52:17.662973 (+ 150us) server_negotiation.cc:299] Negotiation successful
0806 22:52:17.663064 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260806 22:52:17.663174 11311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:17.656202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.253:37573 (local address 127.0.0.1:45470)
0806 22:52:17.656252 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0806 22:52:17.656262 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:52:17.656303 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:52:17.656443 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:52:17.656444 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:52:17.656631 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:52:17.656712 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:52:17.656715 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:52:17.657423 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:52:17.657424 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:52:17.658244 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:52:17.658247 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:52:17.658593 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:52:17.658594 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:52:17.658668 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:52:17.659187 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:52:17.659193 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:52:17.660492 (+ 1299us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:52:17.662104 (+ 1612us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:52:17.662105 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:52:17.662106 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:52:17.662277 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:52:17.662537 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:52:17.662538 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:52:17.662538 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:52:17.662615 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:52:17.662832 (+ 217us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:52:17.662833 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:52:17.662880 (+ 47us) client_negotiation.cc:769] Sending connection context
0806 22:52:17.662958 (+ 78us) client_negotiation.cc:240] Negotiation successful
0806 22:52:17.663052 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260806 22:52:17.663651 11308 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.218.252" port: 36357 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:52:17.664021 11311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:17.663895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.252:36357 (local address 127.0.0.1:50858)
0806 22:52:17.663962 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0806 22:52:17.663996 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.252:36357: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260806 22:52:17.664357 11311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:17.664266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.252:36357 (local address 127.0.0.1:50864)
0806 22:52:17.664306 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0806 22:52:17.664333 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.252:36357: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260806 22:52:17.664445 11308 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.218.252:36357: Network error: Client connection negotiation failed: client connection to 127.28.218.252:36357: connect: Connection refused (error 111)
I20260806 22:52:17.695583 11308 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.218.252" port: 36357 } attempt: 1
I20260806 22:52:17.696059 11311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:17.695902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.252:36357 (local address 127.0.0.1:50878)
0806 22:52:17.695975 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0806 22:52:17.696023 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.252:36357: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260806 22:52:17.696391 11311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:17.696299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.252:36357 (local address 127.0.0.1:50892)
0806 22:52:17.696341 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0806 22:52:17.696369 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.252:36357: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260806 22:52:17.696501 11308 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.218.252:36357: Network error: Client connection negotiation failed: client connection to 127.28.218.252:36357: connect: Connection refused (error 111)
W20260806 22:52:17.697000 11395 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:52:17.697209 11395 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:52:17.697252 11395 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:52:17.698328 11395 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:52:17.698390 11395 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:52:17.698415 11395 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:52:17.698441 11395 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:52:17.698468 11395 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:52:17.699972 11395 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:36525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056586788339-29547-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056586788339-29547-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.28.218.254:44059,127.28.218.253:37573,127.28.218.252:36357
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.252.keytab
--principal=kudu/127.28.218.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.218.252:36357
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056586788339-29547-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.28.218.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11395
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056586788339-29547-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:52:17.700251 11395 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:52:17.700459 11395 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:52:17.705046 11401 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:52:17.705072 11404 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:52:17.705155 11395 server_base.cc:1034] running on GCE node
W20260806 22:52:17.705072 11402 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:52:17.705546 11395 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:52:17.705785 11395 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:52:17.706921 11395 hybrid_clock.cc:648] HybridClock initialized: now 1786056737706903 us; error 35 us; skew 500 ppm
Aug 06 22:52:17 dist-test-slave-1xrh krb5kdc[11219](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056737, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:52:17.709434 11395 init.cc:377] Logged in from keytab as kudu/127.28.218.252@KRBTEST.COM (short username kudu)
I20260806 22:52:17.709770 11395 webserver.cc:460] Webserver started at http://127.28.218.252:33271/ using document root <none> and password file <none>
I20260806 22:52:17.709931 11395 fs_manager.cc:359] Metadata directory not provided
I20260806 22:52:17.709968 11395 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:52:17.710062 11395 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:52:17.710952 11395 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056586788339-29547-0/minicluster-data/master-2/data/instance:
uuid: "3e059ed4bac74bbf9f808f76f0f70b1f"
format_stamp: "Formatted at 2026-08-06 22:52:17 on dist-test-slave-1xrh"
server_key: "673b818b9cc0de75a336e364d3992ae2"
server_key_iv: "11f333129de5e1d17ec72c706da795b8"
server_key_version: "encryptionkey@0"
I20260806 22:52:17.711313 11395 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056586788339-29547-0/minicluster-data/master-2/wal/instance:
uuid: "3e059ed4bac74bbf9f808f76f0f70b1f"
format_stamp: "Formatted at 2026-08-06 22:52:17 on dist-test-slave-1xrh"
server_key: "673b818b9cc0de75a336e364d3992ae2"
server_key_iv: "11f333129de5e1d17ec72c706da795b8"
server_key_version: "encryptionkey@0"
I20260806 22:52:17.712452 11395 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:52:17.713074 11411 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:52:17.713229 11395 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:52:17.713306 11395 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056586788339-29547-0/minicluster-data/master-2/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056586788339-29547-0/minicluster-data/master-2/wal
uuid: "3e059ed4bac74bbf9f808f76f0f70b1f"
format_stamp: "Formatted at 2026-08-06 22:52:17 on dist-test-slave-1xrh"
server_key: "673b818b9cc0de75a336e364d3992ae2"
server_key_iv: "11f333129de5e1d17ec72c706da795b8"
server_key_version: "encryptionkey@0"
I20260806 22:52:17.713368 11395 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056586788339-29547-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056586788339-29547-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056586788339-29547-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:52:17.720328 11388 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.218.252" port: 36357 } attempt: 2
I20260806 22:52:17.720695 11390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:17.720569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.252:36357 (local address 127.0.0.1:50894)
0806 22:52:17.720627 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0806 22:52:17.720663 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.252:36357: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260806 22:52:17.721009 11390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:17.720932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.252:36357 (local address 127.0.0.1:50906)
0806 22:52:17.720968 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0806 22:52:17.720993 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.252:36357: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260806 22:52:17.721117 11388 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.218.252:36357: Network error: Client connection negotiation failed: client connection to 127.28.218.252:36357: connect: Connection refused (error 111)
I20260806 22:52:17.724429 11395 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:52:17.726115 11395 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:52:17.726261 11395 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:52:17.730114 11395 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.252:36357
I20260806 22:52:17.730150 11467 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.252:36357 every 8 connection(s)
I20260806 22:52:17.730482 11395 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056586788339-29547-0/minicluster-data/master-2/data/info.pb
I20260806 22:52:17.731310 11468 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:52:17.732757 11468 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.218.254" port: 44059 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:52:17.738063 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 11395
I20260806 22:52:17.738183 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056586788339-29547-0/minicluster-data/master-2/wal/instance
Aug 06 22:52:17 dist-test-slave-1xrh krb5kdc[11219](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056737, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.252@KRBTEST.COM for kudu/127.28.218.254@KRBTEST.COM
I20260806 22:52:17.738404 29547 external_mini_cluster.cc:1428] Setting key 4d11aba1b6eaf45f891cc94ef9b300c8
I20260806 22:52:17.741524 11310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:17.733545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41150 (local address 127.28.218.254:44059)
0806 22:52:17.733606 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0806 22:52:17.733619 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 22:52:17.733787 (+ 168us) server_negotiation.cc:407] Connection header received
0806 22:52:17.734264 (+ 477us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:52:17.734265 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:52:17.734279 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:52:17.734330 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:52:17.734899 (+ 569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:17.735432 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:17.736378 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:17.736576 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:17.736660 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:52:17.738784 (+ 2124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:52:17.738786 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:52:17.738786 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:52:17.738794 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:52:17.740226 (+ 1432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:52:17.740652 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:52:17.740654 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:52:17.740654 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:52:17.740728 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:52:17.740962 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:52:17.740963 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:52:17.740963 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:52:17.741088 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:52:17.741155 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 22:52:17.741399 (+ 244us) server_negotiation.cc:299] Negotiation successful
0806 22:52:17.741466 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260806 22:52:17.741715 11470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:17.733528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:44059 (local address 127.0.0.1:41150)
0806 22:52:17.733682 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0806 22:52:17.733714 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0806 22:52:17.734156 (+ 442us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:52:17.734394 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:52:17.734397 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:52:17.734635 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:52:17.734811 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:52:17.734814 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:52:17.735531 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:52:17.735532 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:52:17.736293 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:52:17.736295 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:52:17.736666 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:52:17.736668 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:52:17.736784 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:52:17.737193 (+ 409us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:52:17.737200 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:52:17.738663 (+ 1463us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:52:17.740296 (+ 1633us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:52:17.740301 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:52:17.740302 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:52:17.740581 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:52:17.740817 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:52:17.740818 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:52:17.740818 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:52:17.740904 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:52:17.741149 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:52:17.741153 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:52:17.741266 (+ 113us) client_negotiation.cc:769] Sending connection context
0806 22:52:17.741367 (+ 101us) client_negotiation.cc:240] Negotiation successful
0806 22:52:17.741501 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":69,"threads_started":1}
I20260806 22:52:17.742205 11468 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.218.253" port: 37573 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 22:52:17 dist-test-slave-1xrh krb5kdc[11219](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056737, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.218.252@KRBTEST.COM
Aug 06 22:52:17 dist-test-slave-1xrh krb5kdc[11219](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056737, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.252@KRBTEST.COM for kudu/127.28.218.253@KRBTEST.COM
I20260806 22:52:17.746971 11472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:17.738755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50922 (local address 127.28.218.252:36357)
0806 22:52:17.738928 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0806 22:52:17.738932 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:52:17.738995 (+ 63us) server_negotiation.cc:407] Connection header received
0806 22:52:17.739030 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:52:17.739032 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:52:17.739044 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:52:17.739109 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:52:17.739667 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:17.740370 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:17.741284 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:17.741678 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:17.741747 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:52:17.744022 (+ 2275us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:52:17.744029 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:52:17.744030 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:52:17.744037 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:52:17.745759 (+ 1722us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:52:17.746096 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:52:17.746112 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:52:17.746113 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:52:17.746172 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:52:17.746380 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:52:17.746382 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:52:17.746382 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:52:17.746551 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:52:17.746693 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0806 22:52:17.746795 (+ 102us) server_negotiation.cc:299] Negotiation successful
0806 22:52:17.746880 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":95,"threads_started":1}
I20260806 22:52:17.748636 11470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:17.742438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.253:37573 (local address 127.0.0.1:45482)
0806 22:52:17.742481 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0806 22:52:17.742491 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:52:17.742524 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:52:17.742678 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:52:17.742679 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:52:17.742806 (+ 127us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:52:17.742851 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:52:17.742852 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:52:17.743501 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:52:17.743503 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:52:17.744049 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:52:17.744051 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:52:17.744400 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:52:17.744401 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:52:17.744468 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:52:17.744823 (+ 355us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:52:17.744827 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:52:17.746046 (+ 1219us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:52:17.747751 (+ 1705us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:52:17.747752 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:52:17.747752 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:52:17.747966 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:52:17.748160 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:52:17.748161 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:52:17.748161 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:52:17.748211 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:52:17.748449 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:52:17.748450 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:52:17.748480 (+ 30us) client_negotiation.cc:769] Sending connection context
0806 22:52:17.748533 (+ 53us) client_negotiation.cc:240] Negotiation successful
0806 22:52:17.748587 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260806 22:52:17.748711 11392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:17.742512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45482 (local address 127.28.218.253:37573)
0806 22:52:17.742564 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0806 22:52:17.742566 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:52:17.742574 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:52:17.742597 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:52:17.742598 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:52:17.742607 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:52:17.742663 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:52:17.742931 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:17.743429 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:17.744131 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:17.744334 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:17.744403 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:52:17.746155 (+ 1752us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:52:17.746155 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:52:17.746156 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:52:17.746161 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:52:17.747669 (+ 1508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:52:17.748044 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:52:17.748046 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:52:17.748046 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:52:17.748105 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:52:17.748273 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:52:17.748274 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:52:17.748275 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:52:17.748381 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:52:17.748457 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 22:52:17.748557 (+ 100us) server_negotiation.cc:299] Negotiation successful
0806 22:52:17.748628 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 22:52:17.749100 11468 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.218.252" port: 36357 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 22:52:17 dist-test-slave-1xrh krb5kdc[11219](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056737, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.218.254@KRBTEST.COM
Aug 06 22:52:17 dist-test-slave-1xrh krb5kdc[11219](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056737, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.252@KRBTEST.COM for kudu/127.28.218.252@KRBTEST.COM
I20260806 22:52:17.755090 11310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:17.747871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41162 (local address 127.28.218.254:44059)
0806 22:52:17.747925 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0806 22:52:17.747927 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:52:17.747960 (+ 33us) server_negotiation.cc:407] Connection header received
0806 22:52:17.748011 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:52:17.748012 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:52:17.748022 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:52:17.748073 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:52:17.748424 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:17.749023 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:17.749833 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:17.750008 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:17.750085 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:52:17.752343 (+ 2258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:52:17.752344 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:52:17.752345 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:52:17.752352 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:52:17.753859 (+ 1507us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:52:17.754311 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:52:17.754312 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:52:17.754312 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:52:17.754381 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:52:17.754566 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:52:17.754567 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:52:17.754568 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:52:17.754673 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:52:17.754854 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0806 22:52:17.754891 (+ 37us) server_negotiation.cc:299] Negotiation successful
0806 22:52:17.754966 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260806 22:52:17.755329 11470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:17.749296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.252:36357 (local address 127.0.0.1:50930)
0806 22:52:17.749333 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0806 22:52:17.749342 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:52:17.749378 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:52:17.749481 (+ 103us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:52:17.749481 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:52:17.749579 (+ 98us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:52:17.749602 (+ 23us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:52:17.749603 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:52:17.750359 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:52:17.750360 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:52:17.751187 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:52:17.751190 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:52:17.751566 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:52:17.751566 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:52:17.751625 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:52:17.751965 (+ 340us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:52:17.751968 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:52:17.752952 (+ 984us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:52:17.754371 (+ 1419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:52:17.754371 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:52:17.754372 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:52:17.754506 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:52:17.754692 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:52:17.754693 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:52:17.754694 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:52:17.754746 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:52:17.754989 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:52:17.754990 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:52:17.755024 (+ 34us) client_negotiation.cc:769] Sending connection context
0806 22:52:17.755099 (+ 75us) client_negotiation.cc:240] Negotiation successful
0806 22:52:17.755235 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25}
I20260806 22:52:17.755446 11472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:17.749331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50930 (local address 127.28.218.252:36357)
0806 22:52:17.749370 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0806 22:52:17.749372 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:52:17.749379 (+ 7us) server_negotiation.cc:407] Connection header received
0806 22:52:17.749419 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:52:17.749419 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:52:17.749427 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:52:17.749479 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:52:17.749657 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:17.750290 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:17.751276 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:17.751479 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:17.751564 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:52:17.753034 (+ 1470us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:52:17.753034 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:52:17.753035 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:52:17.753040 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:52:17.754259 (+ 1219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:52:17.754572 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:52:17.754573 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:52:17.754574 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:52:17.754623 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:52:17.754814 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:52:17.754815 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:52:17.754815 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:52:17.754922 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:52:17.754984 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 22:52:17.755147 (+ 163us) server_negotiation.cc:299] Negotiation successful
0806 22:52:17.755349 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":128,"server-negotiator.queue_time_us":18}
I20260806 22:52:17.756606 11468 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e059ed4bac74bbf9f808f76f0f70b1f: Bootstrap starting.
I20260806 22:52:17.757445 11468 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3e059ed4bac74bbf9f808f76f0f70b1f: Neither blocks nor log segments found. Creating new log.
I20260806 22:52:17.757714 11468 log.cc:824] T 00000000000000000000000000000000 P 3e059ed4bac74bbf9f808f76f0f70b1f: Log is configured to *not* fsync() on all Append() calls
I20260806 22:52:17.758524 11468 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e059ed4bac74bbf9f808f76f0f70b1f: No bootstrap required, opened a new log
Aug 06 22:52:17 dist-test-slave-1xrh krb5kdc[11219](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056737, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.218.253@KRBTEST.COM
I20260806 22:52:17.760257 11468 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3e059ed4bac74bbf9f808f76f0f70b1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "943296ed5a2741558301ed7f21723f05" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 44059 } } peers { permanent_uuid: "fb3ec65dc40846beab22bb7218ab8f56" member_type: VOTER last_known_addr { host: "127.28.218.253" port: 37573 } } peers { permanent_uuid: "3e059ed4bac74bbf9f808f76f0f70b1f" member_type: VOTER last_known_addr { host: "127.28.218.252" port: 36357 } }
I20260806 22:52:17.760399 11468 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3e059ed4bac74bbf9f808f76f0f70b1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:52:17.760448 11468 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3e059ed4bac74bbf9f808f76f0f70b1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e059ed4bac74bbf9f808f76f0f70b1f, State: Initialized, Role: FOLLOWER
I20260806 22:52:17.760578 11468 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3e059ed4bac74bbf9f808f76f0f70b1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "943296ed5a2741558301ed7f21723f05" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 44059 } } peers { permanent_uuid: "fb3ec65dc40846beab22bb7218ab8f56" member_type: VOTER last_known_addr { host: "127.28.218.253" port: 37573 } } peers { permanent_uuid: "3e059ed4bac74bbf9f808f76f0f70b1f" member_type: VOTER last_known_addr { host: "127.28.218.252" port: 36357 } }
I20260806 22:52:17.760885 11468 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3e059ed4bac74bbf9f808f76f0f70b1f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:52:17.761857 11474 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e059ed4bac74bbf9f808f76f0f70b1f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "943296ed5a2741558301ed7f21723f05" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 44059 } } peers { permanent_uuid: "fb3ec65dc40846beab22bb7218ab8f56" member_type: VOTER last_known_addr { host: "127.28.218.253" port: 37573 } } peers { permanent_uuid: "3e059ed4bac74bbf9f808f76f0f70b1f" member_type: VOTER last_known_addr { host: "127.28.218.252" port: 36357 } } }
I20260806 22:52:17.761957 11474 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e059ed4bac74bbf9f808f76f0f70b1f [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:52:17.762579 11392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:17.755694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45496 (local address 127.28.218.253:37573)
0806 22:52:17.755726 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0806 22:52:17.755728 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:52:17.755735 (+ 7us) server_negotiation.cc:407] Connection header received
0806 22:52:17.755757 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:52:17.755757 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:52:17.755766 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:52:17.755819 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:52:17.756248 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:17.756706 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:17.757303 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:17.757518 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:17.757573 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:52:17.759860 (+ 2287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:52:17.759861 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:52:17.759862 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:52:17.759868 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:52:17.761367 (+ 1499us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:52:17.761695 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:52:17.761696 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:52:17.761697 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:52:17.761753 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:52:17.761952 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:52:17.761954 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:52:17.761954 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:52:17.762101 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:52:17.762326 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0806 22:52:17.762368 (+ 42us) server_negotiation.cc:299] Negotiation successful
0806 22:52:17.762449 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
W20260806 22:52:17.763274 11485 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3e059ed4bac74bbf9f808f76f0f70b1f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 22:52:17.763622 11485 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 22:52:17.764851 29547 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:52:17.764406 11395 master_runner.cc:186] Error getting master registration for 127.28.218.254:44059: OK, instance_id { permanent_uuid: "943296ed5a2741558301ed7f21723f05" instance_seqno: 1786056737428790 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 22:52:17.767331 11395 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 22:52:17.769891 11474 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3e059ed4bac74bbf9f808f76f0f70b1f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:52:17.770012 11474 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3e059ed4bac74bbf9f808f76f0f70b1f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "943296ed5a2741558301ed7f21723f05" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 44059 } } peers { permanent_uuid: "fb3ec65dc40846beab22bb7218ab8f56" member_type: VOTER last_known_addr { host: "127.28.218.253" port: 37573 } } peers { permanent_uuid: "3e059ed4bac74bbf9f808f76f0f70b1f" member_type: VOTER last_known_addr { host: "127.28.218.252" port: 36357 } }
I20260806 22:52:17.770287 11474 leader_election.cc:288] T 00000000000000000000000000000000 P 3e059ed4bac74bbf9f808f76f0f70b1f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 943296ed5a2741558301ed7f21723f05 (127.28.218.254:44059), fb3ec65dc40846beab22bb7218ab8f56 (127.28.218.253:37573)
I20260806 22:52:17.771642 11308 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.218.252" port: 36357 } attempt: 2
I20260806 22:52:17.775000 11310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:17.766117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41172 (local address 127.28.218.254:44059)
0806 22:52:17.766157 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 22:52:17.766159 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:52:17.766260 (+ 101us) server_negotiation.cc:407] Connection header received
0806 22:52:17.766370 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:52:17.766371 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:52:17.766382 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:52:17.766414 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:52:17.767316 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:17.768051 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:17.769049 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:17.769348 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:17.769419 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:52:17.770909 (+ 1490us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:52:17.770910 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:52:17.770911 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:52:17.770918 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:52:17.772921 (+ 2003us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:52:17.773298 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:52:17.773299 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:52:17.773300 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:52:17.773357 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:52:17.774382 (+ 1025us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:52:17.774383 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:52:17.774383 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:52:17.774534 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:52:17.774630 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0806 22:52:17.774792 (+ 162us) server_negotiation.cc:299] Negotiation successful
0806 22:52:17.774889 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 22:52:17.777822 11392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:17.766419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45506 (local address 127.28.218.253:37573)
0806 22:52:17.766451 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0806 22:52:17.766452 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:52:17.766819 (+ 367us) server_negotiation.cc:407] Connection header received
0806 22:52:17.766931 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:52:17.766932 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:52:17.766943 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:52:17.766975 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:52:17.767784 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:17.768442 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:17.769282 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:17.769580 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:17.769645 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:52:17.773145 (+ 3500us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:52:17.773146 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:52:17.773146 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:52:17.773153 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:52:17.775681 (+ 2528us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:52:17.776060 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:52:17.776061 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:52:17.776061 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:52:17.776143 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:52:17.776906 (+ 763us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:52:17.776907 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:52:17.776907 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:52:17.777044 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:52:17.777122 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 22:52:17.777656 (+ 534us) server_negotiation.cc:299] Negotiation successful
0806 22:52:17.777733 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260806 22:52:17.778105 11472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:17.766588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50946 (local address 127.28.218.252:36357)
0806 22:52:17.766626 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0806 22:52:17.766628 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:52:17.767220 (+ 592us) server_negotiation.cc:407] Connection header received
0806 22:52:17.769369 (+ 2149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:52:17.769370 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:52:17.769379 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:52:17.769406 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:52:17.770150 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:17.770778 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:17.771801 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:17.772096 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:17.772138 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:52:17.774390 (+ 2252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:52:17.774391 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:52:17.774391 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:52:17.774398 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:52:17.776047 (+ 1649us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:52:17.776425 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:52:17.776426 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:52:17.776426 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:52:17.776482 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:52:17.777173 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:52:17.777174 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:52:17.777175 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:52:17.777330 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:52:17.777381 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 22:52:17.777911 (+ 530us) server_negotiation.cc:299] Negotiation successful
0806 22:52:17.777989 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
Aug 06 22:52:17 dist-test-slave-1xrh krb5kdc[11219](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056737, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.254@KRBTEST.COM for kudu/127.28.218.252@KRBTEST.COM
I20260806 22:52:17.782226 11311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:17.771854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.252:36357 (local address 127.0.0.1:50952)
0806 22:52:17.771934 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0806 22:52:17.771943 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:52:17.771975 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:52:17.772513 (+ 538us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:52:17.772514 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:52:17.772697 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:52:17.772729 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:52:17.772731 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:52:17.774046 (+ 1315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:52:17.774047 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:52:17.774846 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:52:17.774849 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:52:17.777236 (+ 2387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:52:17.777237 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:52:17.777297 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:52:17.777985 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:52:17.777990 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:52:17.779454 (+ 1464us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:52:17.781296 (+ 1842us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:52:17.781298 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:52:17.781299 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:52:17.781495 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:52:17.781673 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:52:17.781674 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:52:17.781674 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:52:17.781730 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:52:17.781939 (+ 209us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:52:17.781940 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:52:17.781996 (+ 56us) client_negotiation.cc:769] Sending connection context
0806 22:52:17.782058 (+ 62us) client_negotiation.cc:240] Negotiation successful
0806 22:52:17.782138 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":68}
I20260806 22:52:17.782374 11494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:17.772036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50952 (local address 127.28.218.252:36357)
0806 22:52:17.772263 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0806 22:52:17.772264 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:52:17.772308 (+ 44us) server_negotiation.cc:407] Connection header received
0806 22:52:17.772327 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:52:17.772328 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:52:17.772386 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:52:17.772506 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:52:17.772789 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:17.773959 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:17.775181 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:17.777161 (+ 1980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:17.777222 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:52:17.779538 (+ 2316us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:52:17.779540 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:52:17.779540 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:52:17.779548 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:52:17.781221 (+ 1673us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:52:17.781553 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:52:17.781554 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:52:17.781554 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:52:17.781624 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:52:17.781783 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:52:17.781783 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:52:17.781783 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:52:17.781890 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:52:17.781945 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 22:52:17.782090 (+ 145us) server_negotiation.cc:299] Negotiation successful
0806 22:52:17.782206 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":209,"thread_start_us":86,"threads_started":1}
I20260806 22:52:17.784310 11308 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 943296ed5a2741558301ed7f21723f05: Bootstrap starting.
I20260806 22:52:17.785050 11308 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 943296ed5a2741558301ed7f21723f05: Neither blocks nor log segments found. Creating new log.
I20260806 22:52:17.785377 11308 log.cc:824] T 00000000000000000000000000000000 P 943296ed5a2741558301ed7f21723f05: Log is configured to *not* fsync() on all Append() calls
I20260806 22:52:17.786233 11308 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 943296ed5a2741558301ed7f21723f05: No bootstrap required, opened a new log
I20260806 22:52:17.788301 11308 raft_consensus.cc:348] T 00000000000000000000000000000000 P 943296ed5a2741558301ed7f21723f05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "943296ed5a2741558301ed7f21723f05" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 44059 } } peers { permanent_uuid: "fb3ec65dc40846beab22bb7218ab8f56" member_type: VOTER last_known_addr { host: "127.28.218.253" port: 37573 } } peers { permanent_uuid: "3e059ed4bac74bbf9f808f76f0f70b1f" member_type: VOTER last_known_addr { host: "127.28.218.252" port: 36357 } }
I20260806 22:52:17.788466 11308 raft_consensus.cc:374] T 00000000000000000000000000000000 P 943296ed5a2741558301ed7f21723f05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:52:17.788515 11308 raft_consensus.cc:729] T 00000000000000000000000000000000 P 943296ed5a2741558301ed7f21723f05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 943296ed5a2741558301ed7f21723f05, State: Initialized, Role: FOLLOWER
I20260806 22:52:17.788637 11308 consensus_queue.cc:260] T 00000000000000000000000000000000 P 943296ed5a2741558301ed7f21723f05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "943296ed5a2741558301ed7f21723f05" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 44059 } } peers { permanent_uuid: "fb3ec65dc40846beab22bb7218ab8f56" member_type: VOTER last_known_addr { host: "127.28.218.253" port: 37573 } } peers { permanent_uuid: "3e059ed4bac74bbf9f808f76f0f70b1f" member_type: VOTER last_known_addr { host: "127.28.218.252" port: 36357 } }
I20260806 22:52:17.788933 11496 sys_catalog.cc:455] T 00000000000000000000000000000000 P 943296ed5a2741558301ed7f21723f05 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "943296ed5a2741558301ed7f21723f05" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 44059 } } peers { permanent_uuid: "fb3ec65dc40846beab22bb7218ab8f56" member_type: VOTER last_known_addr { host: "127.28.218.253" port: 37573 } } peers { permanent_uuid: "3e059ed4bac74bbf9f808f76f0f70b1f" member_type: VOTER last_known_addr { host: "127.28.218.252" port: 36357 } } }
I20260806 22:52:17.788950 11308 sys_catalog.cc:564] T 00000000000000000000000000000000 P 943296ed5a2741558301ed7f21723f05 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:52:17.789103 11496 sys_catalog.cc:458] T 00000000000000000000000000000000 P 943296ed5a2741558301ed7f21723f05 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:52:17.789357 11280 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3e059ed4bac74bbf9f808f76f0f70b1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "943296ed5a2741558301ed7f21723f05" is_pre_election: true
I20260806 22:52:17.789489 11280 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 943296ed5a2741558301ed7f21723f05 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e059ed4bac74bbf9f808f76f0f70b1f in term 0.
I20260806 22:52:17.789716 11414 leader_election.cc:302] T 00000000000000000000000000000000 P 3e059ed4bac74bbf9f808f76f0f70b1f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e059ed4bac74bbf9f808f76f0f70b1f, 943296ed5a2741558301ed7f21723f05; no voters:
I20260806 22:52:17.789848 11474 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3e059ed4bac74bbf9f808f76f0f70b1f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:52:17.789893 11474 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3e059ed4bac74bbf9f808f76f0f70b1f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:52:17.789922 11474 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3e059ed4bac74bbf9f808f76f0f70b1f [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:52:17.790716 11474 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3e059ed4bac74bbf9f808f76f0f70b1f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "943296ed5a2741558301ed7f21723f05" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 44059 } } peers { permanent_uuid: "fb3ec65dc40846beab22bb7218ab8f56" member_type: VOTER last_known_addr { host: "127.28.218.253" port: 37573 } } peers { permanent_uuid: "3e059ed4bac74bbf9f808f76f0f70b1f" member_type: VOTER last_known_addr { host: "127.28.218.252" port: 36357 } }
I20260806 22:52:17.790845 11474 leader_election.cc:288] T 00000000000000000000000000000000 P 3e059ed4bac74bbf9f808f76f0f70b1f [CANDIDATE]: Term 1 election: Requested vote from peers 943296ed5a2741558301ed7f21723f05 (127.28.218.254:44059), fb3ec65dc40846beab22bb7218ab8f56 (127.28.218.253:37573)
I20260806 22:52:17.791052 11280 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3e059ed4bac74bbf9f808f76f0f70b1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "943296ed5a2741558301ed7f21723f05"
I20260806 22:52:17.791115 11280 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 943296ed5a2741558301ed7f21723f05 [term 0 FOLLOWER]: Advancing to term 1
W20260806 22:52:17.791836 11508 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 943296ed5a2741558301ed7f21723f05: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260806 22:52:17.791944 11280 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 943296ed5a2741558301ed7f21723f05 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e059ed4bac74bbf9f808f76f0f70b1f in term 1.
W20260806 22:52:17.792002 11508 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 22:52:17.792434 11414 leader_election.cc:302] T 00000000000000000000000000000000 P 3e059ed4bac74bbf9f808f76f0f70b1f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e059ed4bac74bbf9f808f76f0f70b1f, 943296ed5a2741558301ed7f21723f05; no voters:
I20260806 22:52:17.792522 11474 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3e059ed4bac74bbf9f808f76f0f70b1f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:52:17.792668 11474 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3e059ed4bac74bbf9f808f76f0f70b1f [term 1 LEADER]: Becoming Leader. State: Replica: 3e059ed4bac74bbf9f808f76f0f70b1f, State: Running, Role: LEADER
I20260806 22:52:17.792755 11474 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3e059ed4bac74bbf9f808f76f0f70b1f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "943296ed5a2741558301ed7f21723f05" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 44059 } } peers { permanent_uuid: "fb3ec65dc40846beab22bb7218ab8f56" member_type: VOTER last_known_addr { host: "127.28.218.253" port: 37573 } } peers { permanent_uuid: "3e059ed4bac74bbf9f808f76f0f70b1f" member_type: VOTER last_known_addr { host: "127.28.218.252" port: 36357 } }
I20260806 22:52:17.793161 11474 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e059ed4bac74bbf9f808f76f0f70b1f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3e059ed4bac74bbf9f808f76f0f70b1f. Latest consensus state: current_term: 1 leader_uuid: "3e059ed4bac74bbf9f808f76f0f70b1f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "943296ed5a2741558301ed7f21723f05" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 44059 } } peers { permanent_uuid: "fb3ec65dc40846beab22bb7218ab8f56" member_type: VOTER last_known_addr { host: "127.28.218.253" port: 37573 } } peers { permanent_uuid: "3e059ed4bac74bbf9f808f76f0f70b1f" member_type: VOTER last_known_addr { host: "127.28.218.252" port: 36357 } } }
I20260806 22:52:17.793238 11474 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e059ed4bac74bbf9f808f76f0f70b1f [sys.catalog]: This master's current role is: LEADER
I20260806 22:52:17.793450 11513 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:52:17.793603 11513 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:52:17.793220 11233 master_runner.cc:186] Error getting master registration for 127.28.218.253:37573: OK, instance_id { permanent_uuid: "fb3ec65dc40846beab22bb7218ab8f56" instance_seqno: 1786056737577262 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 22:52:17.795645 11233 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 22:52:17.795655 11280 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 943296ed5a2741558301ed7f21723f05 [term 1 FOLLOWER]: Refusing update from remote peer 3e059ed4bac74bbf9f808f76f0f70b1f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:52:17.796458 11510 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3e059ed4bac74bbf9f808f76f0f70b1f [LEADER]: Connected to new peer: Peer: permanent_uuid: "943296ed5a2741558301ed7f21723f05" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 44059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:52:17.797799 11496 sys_catalog.cc:455] T 00000000000000000000000000000000 P 943296ed5a2741558301ed7f21723f05 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3e059ed4bac74bbf9f808f76f0f70b1f. Latest consensus state: current_term: 1 leader_uuid: "3e059ed4bac74bbf9f808f76f0f70b1f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "943296ed5a2741558301ed7f21723f05" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 44059 } } peers { permanent_uuid: "fb3ec65dc40846beab22bb7218ab8f56" member_type: VOTER last_known_addr { host: "127.28.218.253" port: 37573 } } peers { permanent_uuid: "3e059ed4bac74bbf9f808f76f0f70b1f" member_type: VOTER last_known_addr { host: "127.28.218.252" port: 36357 } } }
I20260806 22:52:17.797904 11496 sys_catalog.cc:458] T 00000000000000000000000000000000 P 943296ed5a2741558301ed7f21723f05 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:52:17.799015 11474 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e059ed4bac74bbf9f808f76f0f70b1f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3e059ed4bac74bbf9f808f76f0f70b1f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "943296ed5a2741558301ed7f21723f05" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 44059 } } peers { permanent_uuid: "fb3ec65dc40846beab22bb7218ab8f56" member_type: VOTER last_known_addr { host: "127.28.218.253" port: 37573 } } peers { permanent_uuid: "3e059ed4bac74bbf9f808f76f0f70b1f" member_type: VOTER last_known_addr { host: "127.28.218.252" port: 36357 } } }
I20260806 22:52:17.799563 11496 sys_catalog.cc:455] T 00000000000000000000000000000000 P 943296ed5a2741558301ed7f21723f05 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3e059ed4bac74bbf9f808f76f0f70b1f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "943296ed5a2741558301ed7f21723f05" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 44059 } } peers { permanent_uuid: "fb3ec65dc40846beab22bb7218ab8f56" member_type: VOTER last_known_addr { host: "127.28.218.253" port: 37573 } } peers { permanent_uuid: "3e059ed4bac74bbf9f808f76f0f70b1f" member_type: VOTER last_known_addr { host: "127.28.218.252" port: 36357 } } }
I20260806 22:52:17.799644 11496 sys_catalog.cc:458] T 00000000000000000000000000000000 P 943296ed5a2741558301ed7f21723f05 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:52:17.799731 11513 catalog_manager.cc:1347] Generated new cluster ID: eb5415f9ba2543a086b43e080dc99b77
I20260806 22:52:17.799770 11513 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:52:17.799096 11474 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e059ed4bac74bbf9f808f76f0f70b1f [sys.catalog]: This master's current role is: LEADER
I20260806 22:52:17.803283 11514 mvcc.cc:205] Tried to move back new op lower bound from 7315688398007812096 to 7315688397999865856. Current Snapshot: MvccSnapshot[applied={T|T < 7315688398007812096 or (T in {7315688398007812096})}]
I20260806 22:52:17.805346 11515 mvcc.cc:205] Tried to move back new op lower bound from 7315688398007812096 to 7315688397999865856. Current Snapshot: MvccSnapshot[applied={T|T < 7315688398007812096}]
I20260806 22:52:17.806370 11513 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:52:17.807068 11513 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:52:17.812256 11388 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.218.252" port: 36357 } attempt: 3
Aug 06 22:52:17 dist-test-slave-1xrh krb5kdc[11219](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056737, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.253@KRBTEST.COM for kudu/127.28.218.252@KRBTEST.COM
I20260806 22:52:17.818457 11513 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3e059ed4bac74bbf9f808f76f0f70b1f: Generated new TSK 0
I20260806 22:52:17.818724 11513 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:52:17.821481 11494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:17.812727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50956 (local address 127.28.218.252:36357)
0806 22:52:17.812790 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0806 22:52:17.812792 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:52:17.812798 (+ 6us) server_negotiation.cc:407] Connection header received
0806 22:52:17.812818 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:52:17.812820 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:52:17.812828 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:52:17.812915 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:52:17.813824 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:17.814538 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:17.815678 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:17.816006 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:17.816055 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:52:17.818412 (+ 2357us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:52:17.818413 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:52:17.818413 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:52:17.818420 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:52:17.820334 (+ 1914us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:52:17.820706 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:52:17.820707 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:52:17.820707 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:52:17.820759 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:52:17.820956 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:52:17.820956 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:52:17.820957 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:52:17.821094 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:52:17.821161 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 22:52:17.821323 (+ 162us) server_negotiation.cc:299] Negotiation successful
0806 22:52:17.821401 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260806 22:52:17.821482 11390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:17.812543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.252:36357 (local address 127.0.0.1:50956)
0806 22:52:17.812598 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0806 22:52:17.812613 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 22:52:17.812660 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:52:17.813443 (+ 783us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:52:17.813444 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:52:17.813637 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:52:17.813681 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:52:17.813682 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:52:17.814732 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:52:17.814733 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:52:17.815563 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:52:17.815565 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:52:17.816175 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:52:17.816176 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:52:17.816243 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:52:17.816809 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:52:17.816814 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:52:17.818323 (+ 1509us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:52:17.820424 (+ 2101us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:52:17.820425 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:52:17.820426 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:52:17.820634 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:52:17.820820 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:52:17.820821 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:52:17.820821 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:52:17.820884 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:52:17.821158 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:52:17.821159 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:52:17.821219 (+ 60us) client_negotiation.cc:769] Sending connection context
0806 22:52:17.821285 (+ 66us) client_negotiation.cc:240] Negotiation successful
0806 22:52:17.821388 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260806 22:52:17.823377 11388 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb3ec65dc40846beab22bb7218ab8f56: Bootstrap starting.
I20260806 22:52:17.824154 11388 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fb3ec65dc40846beab22bb7218ab8f56: Neither blocks nor log segments found. Creating new log.
I20260806 22:52:17.824499 11388 log.cc:824] T 00000000000000000000000000000000 P fb3ec65dc40846beab22bb7218ab8f56: Log is configured to *not* fsync() on all Append() calls
I20260806 22:52:17.835883 11388 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb3ec65dc40846beab22bb7218ab8f56: No bootstrap required, opened a new log
I20260806 22:52:17.837996 11388 raft_consensus.cc:348] T 00000000000000000000000000000000 P fb3ec65dc40846beab22bb7218ab8f56 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "943296ed5a2741558301ed7f21723f05" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 44059 } } peers { permanent_uuid: "fb3ec65dc40846beab22bb7218ab8f56" member_type: VOTER last_known_addr { host: "127.28.218.253" port: 37573 } } peers { permanent_uuid: "3e059ed4bac74bbf9f808f76f0f70b1f" member_type: VOTER last_known_addr { host: "127.28.218.252" port: 36357 } }
I20260806 22:52:17.838137 11388 raft_consensus.cc:374] T 00000000000000000000000000000000 P fb3ec65dc40846beab22bb7218ab8f56 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:52:17.838178 11388 raft_consensus.cc:729] T 00000000000000000000000000000000 P fb3ec65dc40846beab22bb7218ab8f56 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb3ec65dc40846beab22bb7218ab8f56, State: Initialized, Role: FOLLOWER
I20260806 22:52:17.838296 11388 consensus_queue.cc:260] T 00000000000000000000000000000000 P fb3ec65dc40846beab22bb7218ab8f56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "943296ed5a2741558301ed7f21723f05" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 44059 } } peers { permanent_uuid: "fb3ec65dc40846beab22bb7218ab8f56" member_type: VOTER last_known_addr { host: "127.28.218.253" port: 37573 } } peers { permanent_uuid: "3e059ed4bac74bbf9f808f76f0f70b1f" member_type: VOTER last_known_addr { host: "127.28.218.252" port: 36357 } }
I20260806 22:52:17.838621 11388 sys_catalog.cc:564] T 00000000000000000000000000000000 P fb3ec65dc40846beab22bb7218ab8f56 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:52:17.838709 11356 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fb3ec65dc40846beab22bb7218ab8f56 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:52:17.840122 11358 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3e059ed4bac74bbf9f808f76f0f70b1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb3ec65dc40846beab22bb7218ab8f56" is_pre_election: true
I20260806 22:52:17.840246 11356 raft_consensus.cc:1261] T 00000000000000000000000000000000 P fb3ec65dc40846beab22bb7218ab8f56 [term 1 FOLLOWER]: Refusing update from remote peer 3e059ed4bac74bbf9f808f76f0f70b1f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:52:17.838581 11525 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb3ec65dc40846beab22bb7218ab8f56 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "943296ed5a2741558301ed7f21723f05" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 44059 } } peers { permanent_uuid: "fb3ec65dc40846beab22bb7218ab8f56" member_type: VOTER last_known_addr { host: "127.28.218.253" port: 37573 } } peers { permanent_uuid: "3e059ed4bac74bbf9f808f76f0f70b1f" member_type: VOTER last_known_addr { host: "127.28.218.252" port: 36357 } } }
I20260806 22:52:17.840513 11525 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb3ec65dc40846beab22bb7218ab8f56 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:52:17.840567 11474 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3e059ed4bac74bbf9f808f76f0f70b1f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb3ec65dc40846beab22bb7218ab8f56" member_type: VOTER last_known_addr { host: "127.28.218.253" port: 37573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:52:17.840745 11357 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3e059ed4bac74bbf9f808f76f0f70b1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb3ec65dc40846beab22bb7218ab8f56"
I20260806 22:52:17.841478 11357 raft_consensus.cc:2454] T 00000000000000000000000000000000 P fb3ec65dc40846beab22bb7218ab8f56 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e059ed4bac74bbf9f808f76f0f70b1f in term 1.
I20260806 22:52:17.842962 11540 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb3ec65dc40846beab22bb7218ab8f56 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3e059ed4bac74bbf9f808f76f0f70b1f. Latest consensus state: current_term: 1 leader_uuid: "3e059ed4bac74bbf9f808f76f0f70b1f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "943296ed5a2741558301ed7f21723f05" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 44059 } } peers { permanent_uuid: "fb3ec65dc40846beab22bb7218ab8f56" member_type: VOTER last_known_addr { host: "127.28.218.253" port: 37573 } } peers { permanent_uuid: "3e059ed4bac74bbf9f808f76f0f70b1f" member_type: VOTER last_known_addr { host: "127.28.218.252" port: 36357 } } }
I20260806 22:52:17.843056 11540 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb3ec65dc40846beab22bb7218ab8f56 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:52:17.843098 11540 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb3ec65dc40846beab22bb7218ab8f56 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3e059ed4bac74bbf9f808f76f0f70b1f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "943296ed5a2741558301ed7f21723f05" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 44059 } } peers { permanent_uuid: "fb3ec65dc40846beab22bb7218ab8f56" member_type: VOTER last_known_addr { host: "127.28.218.253" port: 37573 } } peers { permanent_uuid: "3e059ed4bac74bbf9f808f76f0f70b1f" member_type: VOTER last_known_addr { host: "127.28.218.252" port: 36357 } } }
I20260806 22:52:17.843166 11540 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb3ec65dc40846beab22bb7218ab8f56 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:52:17.844928 11537 catalog_manager.cc:1259] Loaded cluster ID: eb5415f9ba2543a086b43e080dc99b77
I20260806 22:52:17.845023 11537 catalog_manager.cc:1552] T 00000000000000000000000000000000 P fb3ec65dc40846beab22bb7218ab8f56: loading cluster ID for follower catalog manager: success
I20260806 22:52:17.846191 11537 catalog_manager.cc:1574] T 00000000000000000000000000000000 P fb3ec65dc40846beab22bb7218ab8f56: acquiring CA information for follower catalog manager: success
I20260806 22:52:17.846570 11537 catalog_manager.cc:1602] T 00000000000000000000000000000000 P fb3ec65dc40846beab22bb7218ab8f56: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 22:52:18.281988 11474 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e059ed4bac74bbf9f808f76f0f70b1f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3e059ed4bac74bbf9f808f76f0f70b1f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "943296ed5a2741558301ed7f21723f05" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 44059 } } peers { permanent_uuid: "fb3ec65dc40846beab22bb7218ab8f56" member_type: VOTER last_known_addr { host: "127.28.218.253" port: 37573 } } peers { permanent_uuid: "3e059ed4bac74bbf9f808f76f0f70b1f" member_type: VOTER last_known_addr { host: "127.28.218.252" port: 36357 } } }
I20260806 22:52:18.282166 11474 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e059ed4bac74bbf9f808f76f0f70b1f [sys.catalog]: This master's current role is: LEADER
I20260806 22:52:18.792563 11508 catalog_manager.cc:1259] Loaded cluster ID: eb5415f9ba2543a086b43e080dc99b77
I20260806 22:52:18.792636 11508 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 943296ed5a2741558301ed7f21723f05: loading cluster ID for follower catalog manager: success
I20260806 22:52:18.793445 11508 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 943296ed5a2741558301ed7f21723f05: acquiring CA information for follower catalog manager: success
I20260806 22:52:18.793733 11508 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 943296ed5a2741558301ed7f21723f05: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 22:52:18.878935 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 11233
I20260806 22:52:18.885241 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 11315
I20260806 22:52:18.891585 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 11395
2026-08-06T22:52:18Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (2761 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786056586788339-29547-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:52:18 dist-test-slave-1xrh krb5kdc[11548](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:52:18 dist-test-slave-1xrh krb5kdc[11548](info): set up 2 sockets
Aug 06 22:52:18 dist-test-slave-1xrh krb5kdc[11548](info): commencing operation
krb5kdc: starting...
W20260806 22:52:19.955603 29547 mini_kdc.cc:121] Time spent starting KDC: real 1.044s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786056586788339-29547-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786056586788339-29547-0/krb5kdc/test-user.keytab.
Aug 06 22:52:20 dist-test-slave-1xrh krb5kdc[11548](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056740, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:52:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:52:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.218.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.218.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.218.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab.
Entry for principal kudu/127.28.218.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab.
I20260806 22:52:20.082098 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786056586788339-29547-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786056586788339-29547-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786056586788339-29547-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.218.254:43629
--webserver_interface=127.28.218.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab
--principal=kudu/127.28.218.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.218.212:41707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.218.254:43629
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:52:20.164615 11562 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:52:20.164841 11562 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:52:20.164865 11562 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:52:20.166006 11562 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:52:20.166039 11562 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:52:20.166054 11562 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:52:20.166066 11562 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:52:20.166078 11562 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:52:20.167508 11562 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:41707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786056586788339-29547-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786056586788339-29547-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.218.254:43629
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab
--principal=kudu/127.28.218.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.218.254:43629
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.218.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11562
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786056586788339-29547-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:52:20.167708 11562 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:52:20.167883 11562 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:52:20.172688 11569 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:52:20.172726 11571 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:52:20.172693 11568 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:52:20.172950 11562 server_base.cc:1034] running on GCE node
I20260806 22:52:20.173177 11562 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:52:20.173418 11562 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:52:20.174557 11562 hybrid_clock.cc:648] HybridClock initialized: now 1786056740174546 us; error 33 us; skew 500 ppm
Aug 06 22:52:20 dist-test-slave-1xrh krb5kdc[11548](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056740, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:52:20.177104 11562 init.cc:377] Logged in from keytab as kudu/127.28.218.254@KRBTEST.COM (short username kudu)
I20260806 22:52:20.177563 11562 webserver.cc:460] Webserver started at http://127.28.218.254:34347/ using document root <none> and password file <none>
I20260806 22:52:20.177748 11562 fs_manager.cc:359] Metadata directory not provided
I20260806 22:52:20.177793 11562 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:52:20.177886 11562 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:52:20.178905 11562 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786056586788339-29547-0/minicluster-data/master-0/data/instance:
uuid: "fb3525af919c441f8c0a9123e494945c"
format_stamp: "Formatted at 2026-08-06 22:52:20 on dist-test-slave-1xrh"
server_key: "056f6b0de3ea6ea5990032dbfbb806ac"
server_key_iv: "630f8e7068d7d51fc2bf35a3582747b6"
server_key_version: "encryptionkey@0"
I20260806 22:52:20.179268 11562 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786056586788339-29547-0/minicluster-data/master-0/wal/instance:
uuid: "fb3525af919c441f8c0a9123e494945c"
format_stamp: "Formatted at 2026-08-06 22:52:20 on dist-test-slave-1xrh"
server_key: "056f6b0de3ea6ea5990032dbfbb806ac"
server_key_iv: "630f8e7068d7d51fc2bf35a3582747b6"
server_key_version: "encryptionkey@0"
I20260806 22:52:20.180562 11562 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:52:20.181282 11578 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:52:20.181492 11562 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:52:20.181593 11562 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786056586788339-29547-0/minicluster-data/master-0/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786056586788339-29547-0/minicluster-data/master-0/wal
uuid: "fb3525af919c441f8c0a9123e494945c"
format_stamp: "Formatted at 2026-08-06 22:52:20 on dist-test-slave-1xrh"
server_key: "056f6b0de3ea6ea5990032dbfbb806ac"
server_key_iv: "630f8e7068d7d51fc2bf35a3582747b6"
server_key_version: "encryptionkey@0"
I20260806 22:52:20.181666 11562 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786056586788339-29547-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786056586788339-29547-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786056586788339-29547-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:52:20.189656 11562 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:52:20.191828 11562 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:52:20.191972 11562 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:52:20.196038 11562 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.254:43629
I20260806 22:52:20.196099 11631 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.254:43629 every 8 connection(s)
I20260806 22:52:20.196435 11562 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
I20260806 22:52:20.197288 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 11562
I20260806 22:52:20.197407 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786056586788339-29547-0/minicluster-data/master-0/wal/instance
I20260806 22:52:20.197409 11632 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:52:20.197654 29547 external_mini_cluster.cc:1428] Setting key 2f454127c9c0448fb32a18f1d1922c86
I20260806 22:52:20.200678 11632 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb3525af919c441f8c0a9123e494945c: Bootstrap starting.
I20260806 22:52:20.201488 11632 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fb3525af919c441f8c0a9123e494945c: Neither blocks nor log segments found. Creating new log.
I20260806 22:52:20.201792 11632 log.cc:824] T 00000000000000000000000000000000 P fb3525af919c441f8c0a9123e494945c: Log is configured to *not* fsync() on all Append() calls
I20260806 22:52:20.202512 11632 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb3525af919c441f8c0a9123e494945c: No bootstrap required, opened a new log
Aug 06 22:52:20 dist-test-slave-1xrh krb5kdc[11548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056740, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.218.254@KRBTEST.COM
I20260806 22:52:20.205084 11632 raft_consensus.cc:348] T 00000000000000000000000000000000 P fb3525af919c441f8c0a9123e494945c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb3525af919c441f8c0a9123e494945c" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 43629 } }
I20260806 22:52:20.205264 11632 raft_consensus.cc:374] T 00000000000000000000000000000000 P fb3525af919c441f8c0a9123e494945c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:52:20.205314 11632 raft_consensus.cc:729] T 00000000000000000000000000000000 P fb3525af919c441f8c0a9123e494945c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb3525af919c441f8c0a9123e494945c, State: Initialized, Role: FOLLOWER
I20260806 22:52:20.205461 11632 consensus_queue.cc:260] T 00000000000000000000000000000000 P fb3525af919c441f8c0a9123e494945c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb3525af919c441f8c0a9123e494945c" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 43629 } }
I20260806 22:52:20.205559 11632 raft_consensus.cc:388] T 00000000000000000000000000000000 P fb3525af919c441f8c0a9123e494945c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:52:20.205605 11632 raft_consensus.cc:482] T 00000000000000000000000000000000 P fb3525af919c441f8c0a9123e494945c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:52:20.205664 11632 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fb3525af919c441f8c0a9123e494945c [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:52:20.206369 11632 raft_consensus.cc:504] T 00000000000000000000000000000000 P fb3525af919c441f8c0a9123e494945c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb3525af919c441f8c0a9123e494945c" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 43629 } }
I20260806 22:52:20.206498 11632 leader_election.cc:302] T 00000000000000000000000000000000 P fb3525af919c441f8c0a9123e494945c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fb3525af919c441f8c0a9123e494945c; no voters:
I20260806 22:52:20.206693 11632 leader_election.cc:288] T 00000000000000000000000000000000 P fb3525af919c441f8c0a9123e494945c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:52:20.206727 11637 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fb3525af919c441f8c0a9123e494945c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:52:20.206858 11637 raft_consensus.cc:686] T 00000000000000000000000000000000 P fb3525af919c441f8c0a9123e494945c [term 1 LEADER]: Becoming Leader. State: Replica: fb3525af919c441f8c0a9123e494945c, State: Running, Role: LEADER
I20260806 22:52:20.206951 11637 consensus_queue.cc:237] T 00000000000000000000000000000000 P fb3525af919c441f8c0a9123e494945c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb3525af919c441f8c0a9123e494945c" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 43629 } }
I20260806 22:52:20.206980 11632 sys_catalog.cc:564] T 00000000000000000000000000000000 P fb3525af919c441f8c0a9123e494945c [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:52:20.207446 11637 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb3525af919c441f8c0a9123e494945c [sys.catalog]: SysCatalogTable state changed. Reason: New leader fb3525af919c441f8c0a9123e494945c. Latest consensus state: current_term: 1 leader_uuid: "fb3525af919c441f8c0a9123e494945c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb3525af919c441f8c0a9123e494945c" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 43629 } } }
I20260806 22:52:20.207535 11637 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb3525af919c441f8c0a9123e494945c [sys.catalog]: This master's current role is: LEADER
I20260806 22:52:20.207463 11638 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb3525af919c441f8c0a9123e494945c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fb3525af919c441f8c0a9123e494945c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb3525af919c441f8c0a9123e494945c" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 43629 } } }
I20260806 22:52:20.207715 11638 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb3525af919c441f8c0a9123e494945c [sys.catalog]: This master's current role is: LEADER
I20260806 22:52:20.207973 11643 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:52:20.208602 11643 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:52:20.209264 11635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:20.198160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53054 (local address 127.28.218.254:43629)
0806 22:52:20.198329 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0806 22:52:20.198336 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:52:20.198361 (+ 25us) server_negotiation.cc:407] Connection header received
0806 22:52:20.198938 (+ 577us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:52:20.198946 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:52:20.199009 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:52:20.199166 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:52:20.199949 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:20.200872 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:20.201963 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:20.202356 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:20.202441 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:52:20.205055 (+ 2614us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:52:20.205059 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:52:20.205059 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:52:20.205073 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:52:20.207039 (+ 1966us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:52:20.207526 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:52:20.207529 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:52:20.207530 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:52:20.207661 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:52:20.207909 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:52:20.207910 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:52:20.207910 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:52:20.208228 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:52:20.208832 (+ 604us) server_negotiation.cc:1035] Waiting for connection context
0806 22:52:20.208906 (+ 74us) server_negotiation.cc:299] Negotiation successful
0806 22:52:20.209048 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":78,"threads_started":1}
I20260806 22:52:20.210294 29547 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:52:20.216450 11643 catalog_manager.cc:1347] Generated new cluster ID: ed868e9d8247469e94d109225cb97615
I20260806 22:52:20.216508 11643 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:52:20.237757 11643 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:52:20.238349 11643 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:52:20.243568 11643 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fb3525af919c441f8c0a9123e494945c: Generated new TSK 0
I20260806 22:52:20.243750 11643 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:52:20.264307 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 11562
2026-08-06T22:52:20Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1373 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:52:20 dist-test-slave-1xrh krb5kdc[11659](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:52:20 dist-test-slave-1xrh krb5kdc[11659](info): set up 2 sockets
Aug 06 22:52:20 dist-test-slave-1xrh krb5kdc[11659](info): commencing operation
krb5kdc: starting...
W20260806 22:52:21.383538 29547 mini_kdc.cc:121] Time spent starting KDC: real 1.099s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/krb5kdc/test-user.keytab.
Aug 06 22:52:21 dist-test-slave-1xrh krb5kdc[11659](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056741, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:52:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:52:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.218.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.218.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.218.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab.
Entry for principal kudu/127.28.218.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab.
I20260806 22:52:21.506042 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.218.254:45957
--webserver_interface=127.28.218.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab
--principal=kudu/127.28.218.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.218.212:41905
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.218.254:45957
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:52:21.591002 11674 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:52:21.591241 11674 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:52:21.591266 11674 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:52:21.592512 11674 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:52:21.592581 11674 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:52:21.592619 11674 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:52:21.592648 11674 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:52:21.592667 11674 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:52:21.594537 11674 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:41905
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.218.254:45957
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab
--principal=kudu/127.28.218.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.218.254:45957
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.218.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11674
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:52:21.594738 11674 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:52:21.594923 11674 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:52:21.599974 11679 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:52:21.599980 11680 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:52:21.600010 11674 server_base.cc:1034] running on GCE node
W20260806 22:52:21.599990 11682 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:52:21.600447 11674 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:52:21.600731 11674 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:52:21.601872 11674 hybrid_clock.cc:648] HybridClock initialized: now 1786056741601852 us; error 40 us; skew 500 ppm
Aug 06 22:52:21 dist-test-slave-1xrh krb5kdc[11659](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056741, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:52:21.604447 11674 init.cc:377] Logged in from keytab as kudu/127.28.218.254@KRBTEST.COM (short username kudu)
I20260806 22:52:21.604949 11674 webserver.cc:460] Webserver started at http://127.28.218.254:33089/ using document root <none> and password file <none>
I20260806 22:52:21.605206 11674 fs_manager.cc:359] Metadata directory not provided
I20260806 22:52:21.605273 11674 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:52:21.605422 11674 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:52:21.606503 11674 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/master-0/data/instance:
uuid: "d312610ebc6a4ab3aaec82261c28aa09"
format_stamp: "Formatted at 2026-08-06 22:52:21 on dist-test-slave-1xrh"
server_key: "299f120946a4497f663707ffd8d20458"
server_key_iv: "375f6ffbd4a558e198668ceaa76da574"
server_key_version: "encryptionkey@0"
I20260806 22:52:21.606866 11674 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/master-0/wal/instance:
uuid: "d312610ebc6a4ab3aaec82261c28aa09"
format_stamp: "Formatted at 2026-08-06 22:52:21 on dist-test-slave-1xrh"
server_key: "299f120946a4497f663707ffd8d20458"
server_key_iv: "375f6ffbd4a558e198668ceaa76da574"
server_key_version: "encryptionkey@0"
I20260806 22:52:21.608268 11674 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:52:21.609020 11689 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:52:21.609206 11674 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:52:21.609309 11674 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/master-0/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/master-0/wal
uuid: "d312610ebc6a4ab3aaec82261c28aa09"
format_stamp: "Formatted at 2026-08-06 22:52:21 on dist-test-slave-1xrh"
server_key: "299f120946a4497f663707ffd8d20458"
server_key_iv: "375f6ffbd4a558e198668ceaa76da574"
server_key_version: "encryptionkey@0"
I20260806 22:52:21.609390 11674 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:52:21.643009 11674 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:52:21.644971 11674 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:52:21.645144 11674 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:52:21.649518 11674 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.254:45957
I20260806 22:52:21.649559 11746 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.254:45957 every 8 connection(s)
I20260806 22:52:21.649945 11674 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
I20260806 22:52:21.650918 11747 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:52:21.651832 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 11674
I20260806 22:52:21.651964 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/master-0/wal/instance
I20260806 22:52:21.652176 29547 external_mini_cluster.cc:1428] Setting key 03b538236c8e63554c1d2dd5f2f82e72
I20260806 22:52:21.653635 11747 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d312610ebc6a4ab3aaec82261c28aa09: Bootstrap starting.
I20260806 22:52:21.654388 11747 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d312610ebc6a4ab3aaec82261c28aa09: Neither blocks nor log segments found. Creating new log.
I20260806 22:52:21.654707 11747 log.cc:824] T 00000000000000000000000000000000 P d312610ebc6a4ab3aaec82261c28aa09: Log is configured to *not* fsync() on all Append() calls
I20260806 22:52:21.655432 11747 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d312610ebc6a4ab3aaec82261c28aa09: No bootstrap required, opened a new log
I20260806 22:52:21.657317 11747 raft_consensus.cc:348] T 00000000000000000000000000000000 P d312610ebc6a4ab3aaec82261c28aa09 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d312610ebc6a4ab3aaec82261c28aa09" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 45957 } }
I20260806 22:52:21.657482 11747 raft_consensus.cc:374] T 00000000000000000000000000000000 P d312610ebc6a4ab3aaec82261c28aa09 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:52:21.657539 11747 raft_consensus.cc:729] T 00000000000000000000000000000000 P d312610ebc6a4ab3aaec82261c28aa09 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d312610ebc6a4ab3aaec82261c28aa09, State: Initialized, Role: FOLLOWER
I20260806 22:52:21.657671 11747 consensus_queue.cc:260] T 00000000000000000000000000000000 P d312610ebc6a4ab3aaec82261c28aa09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d312610ebc6a4ab3aaec82261c28aa09" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 45957 } }
I20260806 22:52:21.657778 11747 raft_consensus.cc:388] T 00000000000000000000000000000000 P d312610ebc6a4ab3aaec82261c28aa09 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:52:21.657821 11747 raft_consensus.cc:482] T 00000000000000000000000000000000 P d312610ebc6a4ab3aaec82261c28aa09 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:52:21.657871 11747 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d312610ebc6a4ab3aaec82261c28aa09 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:52:21.658459 11747 raft_consensus.cc:504] T 00000000000000000000000000000000 P d312610ebc6a4ab3aaec82261c28aa09 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d312610ebc6a4ab3aaec82261c28aa09" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 45957 } }
I20260806 22:52:21.658582 11747 leader_election.cc:302] T 00000000000000000000000000000000 P d312610ebc6a4ab3aaec82261c28aa09 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d312610ebc6a4ab3aaec82261c28aa09; no voters:
I20260806 22:52:21.658749 11747 leader_election.cc:288] T 00000000000000000000000000000000 P d312610ebc6a4ab3aaec82261c28aa09 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:52:21.658774 11752 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d312610ebc6a4ab3aaec82261c28aa09 [term 1 FOLLOWER]: Leader election won for term 1
Aug 06 22:52:21 dist-test-slave-1xrh krb5kdc[11659](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056741, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.218.254@KRBTEST.COM
I20260806 22:52:21.658977 11752 raft_consensus.cc:686] T 00000000000000000000000000000000 P d312610ebc6a4ab3aaec82261c28aa09 [term 1 LEADER]: Becoming Leader. State: Replica: d312610ebc6a4ab3aaec82261c28aa09, State: Running, Role: LEADER
I20260806 22:52:21.659000 11747 sys_catalog.cc:564] T 00000000000000000000000000000000 P d312610ebc6a4ab3aaec82261c28aa09 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:52:21.659070 11752 consensus_queue.cc:237] T 00000000000000000000000000000000 P d312610ebc6a4ab3aaec82261c28aa09 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d312610ebc6a4ab3aaec82261c28aa09" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 45957 } }
I20260806 22:52:21.659412 11754 sys_catalog.cc:455] T 00000000000000000000000000000000 P d312610ebc6a4ab3aaec82261c28aa09 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d312610ebc6a4ab3aaec82261c28aa09" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d312610ebc6a4ab3aaec82261c28aa09" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 45957 } } }
I20260806 22:52:21.659401 11752 sys_catalog.cc:455] T 00000000000000000000000000000000 P d312610ebc6a4ab3aaec82261c28aa09 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d312610ebc6a4ab3aaec82261c28aa09. Latest consensus state: current_term: 1 leader_uuid: "d312610ebc6a4ab3aaec82261c28aa09" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d312610ebc6a4ab3aaec82261c28aa09" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 45957 } } }
I20260806 22:52:21.659555 11754 sys_catalog.cc:458] T 00000000000000000000000000000000 P d312610ebc6a4ab3aaec82261c28aa09 [sys.catalog]: This master's current role is: LEADER
I20260806 22:52:21.659566 11752 sys_catalog.cc:458] T 00000000000000000000000000000000 P d312610ebc6a4ab3aaec82261c28aa09 [sys.catalog]: This master's current role is: LEADER
I20260806 22:52:21.659912 11761 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:52:21.660594 11761 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:52:21.662599 11761 catalog_manager.cc:1347] Generated new cluster ID: 8c5f4590e8804d2dbfed5c3f208eb9e0
I20260806 22:52:21.662658 11761 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:52:21.663754 11750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:21.652709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38298 (local address 127.28.218.254:45957)
0806 22:52:21.652884 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0806 22:52:21.652892 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:52:21.652917 (+ 25us) server_negotiation.cc:407] Connection header received
0806 22:52:21.653406 (+ 489us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:52:21.653412 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:52:21.653464 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:52:21.653575 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:52:21.654385 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:21.655430 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:21.656218 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:21.656655 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:21.656775 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:52:21.659617 (+ 2842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:52:21.659621 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:52:21.659622 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:52:21.659635 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:52:21.661754 (+ 2119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:52:21.662112 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:52:21.662115 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:52:21.662116 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:52:21.662238 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:52:21.662488 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:52:21.662489 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:52:21.662490 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:52:21.662784 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:52:21.662926 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0806 22:52:21.663391 (+ 465us) server_negotiation.cc:299] Negotiation successful
0806 22:52:21.663541 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":85,"threads_started":1}
I20260806 22:52:21.680048 11761 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:52:21.680738 11761 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:52:21.684674 11761 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d312610ebc6a4ab3aaec82261c28aa09: Generated new TSK 0
I20260806 22:52:21.684834 11761 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.28.218.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.218.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.218.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.193.keytab.
Entry for principal kudu/127.28.218.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.193.keytab.
I20260806 22:52:21.719965 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.218.193:0
--local_ip_for_outbound_sockets=127.28.218.193
--webserver_interface=127.28.218.193
--webserver_port=0
--tserver_master_addrs=127.28.218.254:45957
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.193.keytab
--principal=kudu/127.28.218.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.218.212:41905
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:52:21.800254 11774 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:52:21.800451 11774 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:52:21.800473 11774 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:52:21.801574 11774 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:52:21.801615 11774 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:52:21.801666 11774 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.218.193
I20260806 22:52:21.803067 11774 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:41905
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.193.keytab
--principal=kudu/127.28.218.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.218.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.218.193
--webserver_port=0
--tserver_master_addrs=127.28.218.254:45957
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11774
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.218.193
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:52:21.803319 11774 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:52:21.803546 11774 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:52:21.808362 11779 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:52:21.808358 11780 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:52:21.808391 11782 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:52:21.808390 11774 server_base.cc:1034] running on GCE node
I20260806 22:52:21.808732 11774 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:52:21.808909 11774 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:52:21.810051 11774 hybrid_clock.cc:648] HybridClock initialized: now 1786056741810029 us; error 40 us; skew 500 ppm
Aug 06 22:52:21 dist-test-slave-1xrh krb5kdc[11659](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056741, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:52:21.812583 11774 init.cc:377] Logged in from keytab as kudu/127.28.218.193@KRBTEST.COM (short username kudu)
I20260806 22:52:21.813066 11774 webserver.cc:460] Webserver started at http://127.28.218.193:44957/ using document root <none> and password file <none>
I20260806 22:52:21.813333 11774 fs_manager.cc:359] Metadata directory not provided
I20260806 22:52:21.813406 11774 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:52:21.813566 11774 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:52:21.814590 11774 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-0/data/instance:
uuid: "7a4f5f00a15646a3ae17db2711fff3e0"
format_stamp: "Formatted at 2026-08-06 22:52:21 on dist-test-slave-1xrh"
server_key: "649619ee3e06e37c564b970f1879a0a3"
server_key_iv: "d920b093ce9032370d9745a810cb0925"
server_key_version: "encryptionkey@0"
I20260806 22:52:21.814953 11774 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-0/wal/instance:
uuid: "7a4f5f00a15646a3ae17db2711fff3e0"
format_stamp: "Formatted at 2026-08-06 22:52:21 on dist-test-slave-1xrh"
server_key: "649619ee3e06e37c564b970f1879a0a3"
server_key_iv: "d920b093ce9032370d9745a810cb0925"
server_key_version: "encryptionkey@0"
I20260806 22:52:21.816311 11774 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:52:21.817004 11789 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:52:21.817181 11774 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:52:21.817277 11774 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-0/wal
uuid: "7a4f5f00a15646a3ae17db2711fff3e0"
format_stamp: "Formatted at 2026-08-06 22:52:21 on dist-test-slave-1xrh"
server_key: "649619ee3e06e37c564b970f1879a0a3"
server_key_iv: "d920b093ce9032370d9745a810cb0925"
server_key_version: "encryptionkey@0"
I20260806 22:52:21.817374 11774 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:52:21.825678 11774 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:52:21.827896 11774 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:52:21.828059 11774 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:52:21.828269 11774 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:52:21.828559 11774 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:52:21.828612 11774 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:52:21.828647 11774 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:52:21.828675 11774 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:52:21.834403 11774 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.193:34809
I20260806 22:52:21.834455 11907 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.193:34809 every 8 connection(s)
I20260806 22:52:21.834770 11774 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-0/data/info.pb
Aug 06 22:52:21 dist-test-slave-1xrh krb5kdc[11659](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056741, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.193@KRBTEST.COM for kudu/127.28.218.254@KRBTEST.COM
I20260806 22:52:21.843484 11750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:21.835479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.193:37225 (local address 127.28.218.254:45957)
0806 22:52:21.835539 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0806 22:52:21.835541 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:52:21.835722 (+ 181us) server_negotiation.cc:407] Connection header received
0806 22:52:21.836212 (+ 490us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:52:21.836214 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:52:21.836226 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:52:21.836279 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:52:21.837164 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:21.837674 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:21.838636 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:21.838852 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:21.838930 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:52:21.840904 (+ 1974us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:52:21.840906 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:52:21.840906 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:52:21.840913 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:52:21.842081 (+ 1168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:52:21.842473 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:52:21.842475 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:52:21.842475 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:52:21.842535 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:52:21.842771 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:52:21.842772 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:52:21.842772 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:52:21.842883 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:52:21.842959 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 22:52:21.843295 (+ 336us) server_negotiation.cc:299] Negotiation successful
0806 22:52:21.843383 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260806 22:52:21.843638 11910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:21.835454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:45957 (local address 127.28.218.193:37225)
0806 22:52:21.835610 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0806 22:52:21.835642 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0806 22:52:21.836109 (+ 467us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:52:21.836341 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:52:21.836345 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:52:21.836878 (+ 533us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:52:21.837060 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:52:21.837064 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:52:21.837773 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:52:21.837774 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:52:21.838545 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:52:21.838547 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:52:21.838937 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:52:21.838938 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:52:21.839061 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:52:21.839493 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:52:21.839500 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:52:21.840811 (+ 1311us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:52:21.842176 (+ 1365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:52:21.842184 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:52:21.842186 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:52:21.842396 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:52:21.842605 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:52:21.842606 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:52:21.842606 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:52:21.842705 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:52:21.842970 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:52:21.842977 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:52:21.843118 (+ 141us) client_negotiation.cc:769] Sending connection context
0806 22:52:21.843274 (+ 156us) client_negotiation.cc:240] Negotiation successful
0806 22:52:21.843421 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":81,"threads_started":1}
I20260806 22:52:21.844099 11908 heartbeater.cc:347] Connected to a master server at 127.28.218.254:45957
I20260806 22:52:21.844175 11908 heartbeater.cc:464] Registering TS with master...
I20260806 22:52:21.844362 11908 heartbeater.cc:511] Master 127.28.218.254:45957 requested a full tablet report, sending...
I20260806 22:52:21.844600 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 11774
I20260806 22:52:21.844713 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-0/wal/instance
I20260806 22:52:21.844888 11708 ts_manager.cc:194] Registered new tserver with Master: 7a4f5f00a15646a3ae17db2711fff3e0 (127.28.218.193:34809)
I20260806 22:52:21.844908 29547 external_mini_cluster.cc:1428] Setting key 4ebc33c4142cc9567c61bd2532538a89
I20260806 22:52:21.845557 11708 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.218.193@KRBTEST.COM'} at 127.28.218.193:37225
WARNING: no policy specified for kudu/127.28.218.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.218.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.218.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.194.keytab.
Entry for principal kudu/127.28.218.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.194.keytab.
I20260806 22:52:21.870428 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.218.194:0
--local_ip_for_outbound_sockets=127.28.218.194
--webserver_interface=127.28.218.194
--webserver_port=0
--tserver_master_addrs=127.28.218.254:45957
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.194.keytab
--principal=kudu/127.28.218.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.218.212:41905
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:52:21.948019 11913 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:52:21.948204 11913 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:52:21.948225 11913 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:52:21.949240 11913 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:52:21.949273 11913 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:52:21.949297 11913 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.218.194
I20260806 22:52:21.950660 11913 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:41905
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.194.keytab
--principal=kudu/127.28.218.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.218.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.218.194
--webserver_port=0
--tserver_master_addrs=127.28.218.254:45957
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11913
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.218.194
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:52:21.950822 11913 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:52:21.951009 11913 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:52:21.955858 11918 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:52:21.955891 11913 server_base.cc:1034] running on GCE node
W20260806 22:52:21.955864 11919 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:52:21.955873 11921 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:52:21.956254 11913 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:52:21.956498 11913 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:52:21.957655 11913 hybrid_clock.cc:648] HybridClock initialized: now 1786056741957644 us; error 45 us; skew 500 ppm
Aug 06 22:52:21 dist-test-slave-1xrh krb5kdc[11659](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056741, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:52:21.960176 11913 init.cc:377] Logged in from keytab as kudu/127.28.218.194@KRBTEST.COM (short username kudu)
I20260806 22:52:21.960584 11913 webserver.cc:460] Webserver started at http://127.28.218.194:36183/ using document root <none> and password file <none>
I20260806 22:52:21.960839 11913 fs_manager.cc:359] Metadata directory not provided
I20260806 22:52:21.960918 11913 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:52:21.961071 11913 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:52:21.962047 11913 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-1/data/instance:
uuid: "860a5cffbfbc4199a28144ec07d1fcd3"
format_stamp: "Formatted at 2026-08-06 22:52:21 on dist-test-slave-1xrh"
server_key: "ea7cf74de7358c4022b4b16d2b569640"
server_key_iv: "c45dfeff539ec2dd568e793ed682e328"
server_key_version: "encryptionkey@0"
I20260806 22:52:21.962399 11913 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-1/wal/instance:
uuid: "860a5cffbfbc4199a28144ec07d1fcd3"
format_stamp: "Formatted at 2026-08-06 22:52:21 on dist-test-slave-1xrh"
server_key: "ea7cf74de7358c4022b4b16d2b569640"
server_key_iv: "c45dfeff539ec2dd568e793ed682e328"
server_key_version: "encryptionkey@0"
I20260806 22:52:21.963728 11913 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260806 22:52:21.964576 11928 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:52:21.964747 11913 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:52:21.964844 11913 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-1/wal
uuid: "860a5cffbfbc4199a28144ec07d1fcd3"
format_stamp: "Formatted at 2026-08-06 22:52:21 on dist-test-slave-1xrh"
server_key: "ea7cf74de7358c4022b4b16d2b569640"
server_key_iv: "c45dfeff539ec2dd568e793ed682e328"
server_key_version: "encryptionkey@0"
I20260806 22:52:21.964927 11913 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:52:21.983659 11913 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:52:21.985594 11913 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:52:21.985749 11913 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:52:21.985975 11913 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:52:21.986268 11913 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:52:21.986322 11913 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:52:21.986363 11913 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:52:21.986392 11913 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:52:21.991828 11913 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.194:42589
I20260806 22:52:21.991871 12048 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.194:42589 every 8 connection(s)
I20260806 22:52:21.992166 11913 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-1/data/info.pb
I20260806 22:52:21.994786 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 11913
I20260806 22:52:21.994921 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-1/wal/instance
I20260806 22:52:21.995147 29547 external_mini_cluster.cc:1428] Setting key c056dd67cd1fa66a089e9b47017cbc6a
Aug 06 22:52:21 dist-test-slave-1xrh krb5kdc[11659](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056741, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.194@KRBTEST.COM for kudu/127.28.218.254@KRBTEST.COM
I20260806 22:52:22.002339 11750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:21.992822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.194:51363 (local address 127.28.218.254:45957)
0806 22:52:21.992907 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0806 22:52:21.992910 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:52:21.993096 (+ 186us) server_negotiation.cc:407] Connection header received
0806 22:52:21.993608 (+ 512us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:52:21.993609 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:52:21.993622 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:52:21.993678 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:52:21.994644 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:21.995376 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:21.996787 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:21.996974 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:21.997032 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:52:21.999281 (+ 2249us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:52:21.999282 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:52:21.999282 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:52:21.999290 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:52:22.000987 (+ 1697us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:52:22.001375 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:52:22.001376 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:52:22.001377 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:52:22.001440 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:52:22.001681 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:52:22.001683 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:52:22.001683 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:52:22.001831 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:52:22.001909 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 22:52:22.002148 (+ 239us) server_negotiation.cc:299] Negotiation successful
0806 22:52:22.002233 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 22:52:22.002363 12051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:21.992769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:45957 (local address 127.28.218.194:51363)
0806 22:52:21.992976 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0806 22:52:21.993017 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 22:52:21.993503 (+ 486us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:52:21.993741 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:52:21.993745 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:52:21.994354 (+ 609us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:52:21.994539 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:52:21.994544 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:52:21.995473 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:52:21.995474 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:52:21.996707 (+ 1233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:52:21.996711 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:52:21.997067 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:52:21.997068 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:52:21.997208 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:52:21.997780 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:52:21.997792 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:52:21.999191 (+ 1399us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:52:22.001078 (+ 1887us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:52:22.001084 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:52:22.001085 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:52:22.001299 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:52:22.001519 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:52:22.001520 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:52:22.001520 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:52:22.001615 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:52:22.001900 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:52:22.001903 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:52:22.002007 (+ 104us) client_negotiation.cc:769] Sending connection context
0806 22:52:22.002096 (+ 89us) client_negotiation.cc:240] Negotiation successful
0806 22:52:22.002208 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":87,"threads_started":1}
I20260806 22:52:22.002838 12049 heartbeater.cc:347] Connected to a master server at 127.28.218.254:45957
I20260806 22:52:22.002951 12049 heartbeater.cc:464] Registering TS with master...
I20260806 22:52:22.003204 12049 heartbeater.cc:511] Master 127.28.218.254:45957 requested a full tablet report, sending...
I20260806 22:52:22.003654 11708 ts_manager.cc:194] Registered new tserver with Master: 860a5cffbfbc4199a28144ec07d1fcd3 (127.28.218.194:42589)
I20260806 22:52:22.004175 11708 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.218.194@KRBTEST.COM'} at 127.28.218.194:51363
WARNING: no policy specified for kudu/127.28.218.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.218.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.218.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.195.keytab.
Entry for principal kudu/127.28.218.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.195.keytab.
I20260806 22:52:22.022476 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.218.195:0
--local_ip_for_outbound_sockets=127.28.218.195
--webserver_interface=127.28.218.195
--webserver_port=0
--tserver_master_addrs=127.28.218.254:45957
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.195.keytab
--principal=kudu/127.28.218.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.218.212:41905
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:52:22.100710 12054 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:52:22.100920 12054 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:52:22.100965 12054 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:52:22.102036 12054 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:52:22.102101 12054 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:52:22.102144 12054 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.218.195
I20260806 22:52:22.103545 12054 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:41905
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.195.keytab
--principal=kudu/127.28.218.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.218.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.218.195
--webserver_port=0
--tserver_master_addrs=127.28.218.254:45957
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12054
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.218.195
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:52:22.103790 12054 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:52:22.104050 12054 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:52:22.108887 12059 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:52:22.108907 12060 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:52:22.109015 12054 server_base.cc:1034] running on GCE node
W20260806 22:52:22.109093 12062 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:52:22.109325 12054 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:52:22.109555 12054 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:52:22.110689 12054 hybrid_clock.cc:648] HybridClock initialized: now 1786056742110690 us; error 48 us; skew 500 ppm
Aug 06 22:52:22 dist-test-slave-1xrh krb5kdc[11659](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056742, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:52:22.113370 12054 init.cc:377] Logged in from keytab as kudu/127.28.218.195@KRBTEST.COM (short username kudu)
I20260806 22:52:22.113767 12054 webserver.cc:460] Webserver started at http://127.28.218.195:41411/ using document root <none> and password file <none>
I20260806 22:52:22.114118 12054 fs_manager.cc:359] Metadata directory not provided
I20260806 22:52:22.114179 12054 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:52:22.114338 12054 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:52:22.115402 12054 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-2/data/instance:
uuid: "4233903adc864154b6565c00f5654b70"
format_stamp: "Formatted at 2026-08-06 22:52:22 on dist-test-slave-1xrh"
server_key: "cf2c4923bd749fe655cf5a8c0ad4a6c6"
server_key_iv: "e4ba17445af6dc4914b6655b98d54357"
server_key_version: "encryptionkey@0"
I20260806 22:52:22.115738 12054 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-2/wal/instance:
uuid: "4233903adc864154b6565c00f5654b70"
format_stamp: "Formatted at 2026-08-06 22:52:22 on dist-test-slave-1xrh"
server_key: "cf2c4923bd749fe655cf5a8c0ad4a6c6"
server_key_iv: "e4ba17445af6dc4914b6655b98d54357"
server_key_version: "encryptionkey@0"
I20260806 22:52:22.117025 12054 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:52:22.117725 12069 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:52:22.117894 12054 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:52:22.117988 12054 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-2/wal
uuid: "4233903adc864154b6565c00f5654b70"
format_stamp: "Formatted at 2026-08-06 22:52:22 on dist-test-slave-1xrh"
server_key: "cf2c4923bd749fe655cf5a8c0ad4a6c6"
server_key_iv: "e4ba17445af6dc4914b6655b98d54357"
server_key_version: "encryptionkey@0"
I20260806 22:52:22.118062 12054 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:52:22.147643 12054 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:52:22.150058 12054 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:52:22.150214 12054 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:52:22.150455 12054 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:52:22.150756 12054 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:52:22.150810 12054 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:52:22.150843 12054 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:52:22.150888 12054 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:52:22.156893 12054 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.195:45531
I20260806 22:52:22.156936 12189 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.195:45531 every 8 connection(s)
I20260806 22:52:22.157233 12054 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-2/data/info.pb
Aug 06 22:52:22 dist-test-slave-1xrh krb5kdc[11659](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056742, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.195@KRBTEST.COM for kudu/127.28.218.254@KRBTEST.COM
I20260806 22:52:22.166386 11750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:22.158039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.195:60393 (local address 127.28.218.254:45957)
0806 22:52:22.158100 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0806 22:52:22.158102 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:52:22.158141 (+ 39us) server_negotiation.cc:407] Connection header received
0806 22:52:22.158657 (+ 516us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:52:22.158658 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:52:22.158670 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:52:22.158727 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:52:22.159761 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:22.160331 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:22.161393 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:22.161596 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:22.161663 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:52:22.163736 (+ 2073us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:52:22.163737 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:52:22.163737 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:52:22.163744 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:52:22.165248 (+ 1504us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:52:22.165612 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:52:22.165613 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:52:22.165613 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:52:22.165664 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:52:22.165906 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:52:22.165906 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:52:22.165907 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:52:22.166005 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:52:22.166054 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 22:52:22.166267 (+ 213us) server_negotiation.cc:299] Negotiation successful
0806 22:52:22.166325 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 22:52:22.166548 12192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:22.157871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:45957 (local address 127.28.218.195:60393)
0806 22:52:22.158019 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0806 22:52:22.158059 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 22:52:22.158531 (+ 472us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:52:22.158797 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:52:22.158800 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:52:22.159432 (+ 632us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:52:22.159614 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:52:22.159619 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:52:22.160432 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:52:22.160433 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:52:22.161288 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:52:22.161290 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:52:22.161683 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:52:22.161684 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:52:22.161802 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:52:22.162283 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:52:22.162291 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:52:22.163641 (+ 1350us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:52:22.165327 (+ 1686us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:52:22.165333 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:52:22.165334 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:52:22.165546 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:52:22.165722 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:52:22.165723 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:52:22.165723 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:52:22.165851 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:52:22.166058 (+ 207us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:52:22.166062 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:52:22.166166 (+ 104us) client_negotiation.cc:769] Sending connection context
0806 22:52:22.166242 (+ 76us) client_negotiation.cc:240] Negotiation successful
0806 22:52:22.166354 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":82,"threads_started":1}
I20260806 22:52:22.166968 12190 heartbeater.cc:347] Connected to a master server at 127.28.218.254:45957
I20260806 22:52:22.167068 12190 heartbeater.cc:464] Registering TS with master...
I20260806 22:52:22.167249 12190 heartbeater.cc:511] Master 127.28.218.254:45957 requested a full tablet report, sending...
I20260806 22:52:22.167282 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 12054
I20260806 22:52:22.167411 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-2/wal/instance
I20260806 22:52:22.167655 29547 external_mini_cluster.cc:1428] Setting key e5066309975eb5cc7fe570a620fe8cec
I20260806 22:52:22.167656 11708 ts_manager.cc:194] Registered new tserver with Master: 4233903adc864154b6565c00f5654b70 (127.28.218.195:45531)
I20260806 22:52:22.168147 11708 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.218.195@KRBTEST.COM'} at 127.28.218.195:60393
I20260806 22:52:22.168469 29547 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:52:22.174688 11750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:22.169717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38314 (local address 127.28.218.254:45957)
0806 22:52:22.169778 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0806 22:52:22.169780 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:52:22.169850 (+ 70us) server_negotiation.cc:407] Connection header received
0806 22:52:22.169883 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:52:22.169883 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:52:22.169890 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:52:22.169933 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:52:22.170396 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:22.170803 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:22.171560 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:22.171724 (+ 164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:22.171772 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:52:22.172735 (+ 963us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:52:22.172736 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:52:22.172736 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:52:22.172742 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:52:22.173830 (+ 1088us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:52:22.174112 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:52:22.174113 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:52:22.174113 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:52:22.174157 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:52:22.174294 (+ 137us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:52:22.174294 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:52:22.174294 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:52:22.174396 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:52:22.174462 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 22:52:22.174554 (+ 92us) server_negotiation.cc:299] Negotiation successful
0806 22:52:22.174605 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 22:52:22.176337 11707 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38314:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:52:22.177870 11707 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:52:22.184206 12198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:22.179826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.195:45531 (local address 127.0.0.1:54902)
0806 22:52:22.179941 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0806 22:52:22.179983 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 22:52:22.180046 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:52:22.180550 (+ 504us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:52:22.180553 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:52:22.180555 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:52:22.180644 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:52:22.180646 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:52:22.182005 (+ 1359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:52:22.182006 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:52:22.183019 (+ 1013us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:52:22.183028 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:52:22.183927 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:52:22.183928 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:52:22.183993 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:52:22.184015 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 22:52:22.184063 (+ 48us) client_negotiation.cc:240] Negotiation successful
0806 22:52:22.184121 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":90,"mutex_wait_us":78,"thread_start_us":69,"threads_started":1}
I20260806 22:52:22.184262 12203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:22.180168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47114 (local address 127.28.218.193:34809)
0806 22:52:22.180344 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0806 22:52:22.180349 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:52:22.180360 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:52:22.180403 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:52:22.180405 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:52:22.180422 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:52:22.180515 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:52:22.180750 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:22.181895 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:22.183340 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:22.183822 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:22.183913 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:52:22.183967 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 22:52:22.184133 (+ 166us) server_negotiation.cc:299] Negotiation successful
0806 22:52:22.184174 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":106,"threads_started":1}
I20260806 22:52:22.184418 12201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:22.180025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.194:42589 (local address 127.0.0.1:59110)
0806 22:52:22.180586 (+ 561us) negotiation.cc:107] Waiting for socket to connect
0806 22:52:22.180596 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:52:22.180652 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:52:22.180968 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:52:22.180969 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:52:22.180971 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:52:22.181009 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:52:22.181011 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:52:22.181887 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:52:22.181888 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:52:22.183238 (+ 1350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:52:22.183241 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:52:22.184200 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:52:22.184201 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:52:22.184262 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:52:22.184265 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:52:22.184317 (+ 52us) client_negotiation.cc:240] Negotiation successful
0806 22:52:22.184353 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":549,"mutex_wait_us":4,"thread_start_us":38,"threads_started":1}
I20260806 22:52:22.184542 12202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:22.179998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54902 (local address 127.28.218.195:45531)
0806 22:52:22.180387 (+ 389us) server_negotiation.cc:206] Beginning negotiation
0806 22:52:22.180391 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:52:22.180400 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:52:22.180428 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:52:22.180430 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:52:22.180443 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:52:22.180514 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:52:22.180744 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:22.181895 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:22.183108 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:22.183819 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:22.184335 (+ 516us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:52:22.184390 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 22:52:22.184413 (+ 23us) server_negotiation.cc:299] Negotiation successful
0806 22:52:22.184456 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":361,"thread_start_us":79,"threads_started":1}
I20260806 22:52:22.184641 12204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:22.180200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59110 (local address 127.28.218.194:42589)
0806 22:52:22.180376 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0806 22:52:22.180380 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:52:22.180726 (+ 346us) server_negotiation.cc:407] Connection header received
0806 22:52:22.180771 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:52:22.180773 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:52:22.180806 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:52:22.180878 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:52:22.181091 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:22.181786 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:22.183325 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:22.184052 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:22.184117 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:52:22.184166 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 22:52:22.184508 (+ 342us) server_negotiation.cc:299] Negotiation successful
0806 22:52:22.184556 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":111,"threads_started":1}
I20260806 22:52:22.184872 12199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:22.179870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.193:34809 (local address 127.0.0.1:47114)
0806 22:52:22.179970 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0806 22:52:22.179983 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:52:22.180046 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:52:22.180550 (+ 504us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:52:22.180553 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:52:22.180557 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:52:22.180641 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:52:22.180643 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:52:22.182086 (+ 1443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:52:22.182087 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:52:22.183238 (+ 1151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:52:22.183241 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:52:22.183911 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:52:22.183912 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:52:22.184001 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:52:22.184014 (+ 13us) client_negotiation.cc:769] Sending connection context
0806 22:52:22.184115 (+ 101us) client_negotiation.cc:240] Negotiation successful
0806 22:52:22.184158 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92,"mutex_wait_us":42,"thread_start_us":59,"threads_started":1}
I20260806 22:52:22.185176 12121 tablet_service.cc:1467] Processing CreateTablet for tablet 462fd37583e54d5d97eb8ce70939edf7 (DEFAULT_TABLE table=test-table [id=b0ee7b49c0e743dc9751b653532dd0d0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:52:22.185335 11978 tablet_service.cc:1467] Processing CreateTablet for tablet 462fd37583e54d5d97eb8ce70939edf7 (DEFAULT_TABLE table=test-table [id=b0ee7b49c0e743dc9751b653532dd0d0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:52:22.185541 12121 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 462fd37583e54d5d97eb8ce70939edf7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:52:22.185679 11839 tablet_service.cc:1467] Processing CreateTablet for tablet 462fd37583e54d5d97eb8ce70939edf7 (DEFAULT_TABLE table=test-table [id=b0ee7b49c0e743dc9751b653532dd0d0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:52:22.185808 11978 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 462fd37583e54d5d97eb8ce70939edf7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:52:22.185992 11839 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 462fd37583e54d5d97eb8ce70939edf7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:52:22.188669 12205 tablet_bootstrap.cc:492] T 462fd37583e54d5d97eb8ce70939edf7 P 860a5cffbfbc4199a28144ec07d1fcd3: Bootstrap starting.
I20260806 22:52:22.189164 12206 tablet_bootstrap.cc:492] T 462fd37583e54d5d97eb8ce70939edf7 P 4233903adc864154b6565c00f5654b70: Bootstrap starting.
I20260806 22:52:22.189345 12207 tablet_bootstrap.cc:492] T 462fd37583e54d5d97eb8ce70939edf7 P 7a4f5f00a15646a3ae17db2711fff3e0: Bootstrap starting.
I20260806 22:52:22.189647 12205 tablet_bootstrap.cc:654] T 462fd37583e54d5d97eb8ce70939edf7 P 860a5cffbfbc4199a28144ec07d1fcd3: Neither blocks nor log segments found. Creating new log.
I20260806 22:52:22.189833 12206 tablet_bootstrap.cc:654] T 462fd37583e54d5d97eb8ce70939edf7 P 4233903adc864154b6565c00f5654b70: Neither blocks nor log segments found. Creating new log.
I20260806 22:52:22.189977 12207 tablet_bootstrap.cc:654] T 462fd37583e54d5d97eb8ce70939edf7 P 7a4f5f00a15646a3ae17db2711fff3e0: Neither blocks nor log segments found. Creating new log.
I20260806 22:52:22.190259 12205 log.cc:824] T 462fd37583e54d5d97eb8ce70939edf7 P 860a5cffbfbc4199a28144ec07d1fcd3: Log is configured to *not* fsync() on all Append() calls
I20260806 22:52:22.190403 12206 log.cc:824] T 462fd37583e54d5d97eb8ce70939edf7 P 4233903adc864154b6565c00f5654b70: Log is configured to *not* fsync() on all Append() calls
I20260806 22:52:22.190536 12207 log.cc:824] T 462fd37583e54d5d97eb8ce70939edf7 P 7a4f5f00a15646a3ae17db2711fff3e0: Log is configured to *not* fsync() on all Append() calls
I20260806 22:52:22.191082 12206 tablet_bootstrap.cc:492] T 462fd37583e54d5d97eb8ce70939edf7 P 4233903adc864154b6565c00f5654b70: No bootstrap required, opened a new log
I20260806 22:52:22.191145 12205 tablet_bootstrap.cc:492] T 462fd37583e54d5d97eb8ce70939edf7 P 860a5cffbfbc4199a28144ec07d1fcd3: No bootstrap required, opened a new log
I20260806 22:52:22.191181 12206 ts_tablet_manager.cc:1397] T 462fd37583e54d5d97eb8ce70939edf7 P 4233903adc864154b6565c00f5654b70: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:52:22.191206 12205 ts_tablet_manager.cc:1397] T 462fd37583e54d5d97eb8ce70939edf7 P 860a5cffbfbc4199a28144ec07d1fcd3: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 22:52:22.192421 12206 raft_consensus.cc:348] T 462fd37583e54d5d97eb8ce70939edf7 P 4233903adc864154b6565c00f5654b70 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4233903adc864154b6565c00f5654b70" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 45531 } } peers { permanent_uuid: "860a5cffbfbc4199a28144ec07d1fcd3" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 42589 } } peers { permanent_uuid: "7a4f5f00a15646a3ae17db2711fff3e0" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 34809 } }
I20260806 22:52:22.192430 12205 raft_consensus.cc:348] T 462fd37583e54d5d97eb8ce70939edf7 P 860a5cffbfbc4199a28144ec07d1fcd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4233903adc864154b6565c00f5654b70" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 45531 } } peers { permanent_uuid: "860a5cffbfbc4199a28144ec07d1fcd3" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 42589 } } peers { permanent_uuid: "7a4f5f00a15646a3ae17db2711fff3e0" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 34809 } }
I20260806 22:52:22.192561 12205 raft_consensus.cc:374] T 462fd37583e54d5d97eb8ce70939edf7 P 860a5cffbfbc4199a28144ec07d1fcd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:52:22.192565 12206 raft_consensus.cc:374] T 462fd37583e54d5d97eb8ce70939edf7 P 4233903adc864154b6565c00f5654b70 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:52:22.192581 12205 raft_consensus.cc:729] T 462fd37583e54d5d97eb8ce70939edf7 P 860a5cffbfbc4199a28144ec07d1fcd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 860a5cffbfbc4199a28144ec07d1fcd3, State: Initialized, Role: FOLLOWER
I20260806 22:52:22.192615 12206 raft_consensus.cc:729] T 462fd37583e54d5d97eb8ce70939edf7 P 4233903adc864154b6565c00f5654b70 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4233903adc864154b6565c00f5654b70, State: Initialized, Role: FOLLOWER
I20260806 22:52:22.192724 12205 consensus_queue.cc:260] T 462fd37583e54d5d97eb8ce70939edf7 P 860a5cffbfbc4199a28144ec07d1fcd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4233903adc864154b6565c00f5654b70" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 45531 } } peers { permanent_uuid: "860a5cffbfbc4199a28144ec07d1fcd3" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 42589 } } peers { permanent_uuid: "7a4f5f00a15646a3ae17db2711fff3e0" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 34809 } }
I20260806 22:52:22.192730 12206 consensus_queue.cc:260] T 462fd37583e54d5d97eb8ce70939edf7 P 4233903adc864154b6565c00f5654b70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4233903adc864154b6565c00f5654b70" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 45531 } } peers { permanent_uuid: "860a5cffbfbc4199a28144ec07d1fcd3" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 42589 } } peers { permanent_uuid: "7a4f5f00a15646a3ae17db2711fff3e0" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 34809 } }
I20260806 22:52:22.193046 12205 ts_tablet_manager.cc:1428] T 462fd37583e54d5d97eb8ce70939edf7 P 860a5cffbfbc4199a28144ec07d1fcd3: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:52:22.193075 12049 heartbeater.cc:503] Master 127.28.218.254:45957 was elected leader, sending a full tablet report...
I20260806 22:52:22.193064 12206 ts_tablet_manager.cc:1428] T 462fd37583e54d5d97eb8ce70939edf7 P 4233903adc864154b6565c00f5654b70: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:52:22.193074 12190 heartbeater.cc:503] Master 127.28.218.254:45957 was elected leader, sending a full tablet report...
I20260806 22:52:22.193887 12207 tablet_bootstrap.cc:492] T 462fd37583e54d5d97eb8ce70939edf7 P 7a4f5f00a15646a3ae17db2711fff3e0: No bootstrap required, opened a new log
I20260806 22:52:22.193960 12207 ts_tablet_manager.cc:1397] T 462fd37583e54d5d97eb8ce70939edf7 P 7a4f5f00a15646a3ae17db2711fff3e0: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260806 22:52:22.195559 12207 raft_consensus.cc:348] T 462fd37583e54d5d97eb8ce70939edf7 P 7a4f5f00a15646a3ae17db2711fff3e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4233903adc864154b6565c00f5654b70" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 45531 } } peers { permanent_uuid: "860a5cffbfbc4199a28144ec07d1fcd3" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 42589 } } peers { permanent_uuid: "7a4f5f00a15646a3ae17db2711fff3e0" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 34809 } }
I20260806 22:52:22.195675 12207 raft_consensus.cc:374] T 462fd37583e54d5d97eb8ce70939edf7 P 7a4f5f00a15646a3ae17db2711fff3e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:52:22.195696 12207 raft_consensus.cc:729] T 462fd37583e54d5d97eb8ce70939edf7 P 7a4f5f00a15646a3ae17db2711fff3e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a4f5f00a15646a3ae17db2711fff3e0, State: Initialized, Role: FOLLOWER
I20260806 22:52:22.195797 12207 consensus_queue.cc:260] T 462fd37583e54d5d97eb8ce70939edf7 P 7a4f5f00a15646a3ae17db2711fff3e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4233903adc864154b6565c00f5654b70" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 45531 } } peers { permanent_uuid: "860a5cffbfbc4199a28144ec07d1fcd3" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 42589 } } peers { permanent_uuid: "7a4f5f00a15646a3ae17db2711fff3e0" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 34809 } }
I20260806 22:52:22.196024 12207 ts_tablet_manager.cc:1428] T 462fd37583e54d5d97eb8ce70939edf7 P 7a4f5f00a15646a3ae17db2711fff3e0: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:52:22.196099 11908 heartbeater.cc:503] Master 127.28.218.254:45957 was elected leader, sending a full tablet report...
W20260806 22:52:22.242651 12050 tablet.cc:2369] T 462fd37583e54d5d97eb8ce70939edf7 P 860a5cffbfbc4199a28144ec07d1fcd3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 22:52:22.335511 11909 tablet.cc:2369] T 462fd37583e54d5d97eb8ce70939edf7 P 7a4f5f00a15646a3ae17db2711fff3e0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 22:52:22.407748 12191 tablet.cc:2369] T 462fd37583e54d5d97eb8ce70939edf7 P 4233903adc864154b6565c00f5654b70: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:52:22.500903 12211 raft_consensus.cc:482] T 462fd37583e54d5d97eb8ce70939edf7 P 860a5cffbfbc4199a28144ec07d1fcd3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:52:22.501080 12211 raft_consensus.cc:504] T 462fd37583e54d5d97eb8ce70939edf7 P 860a5cffbfbc4199a28144ec07d1fcd3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4233903adc864154b6565c00f5654b70" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 45531 } } peers { permanent_uuid: "860a5cffbfbc4199a28144ec07d1fcd3" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 42589 } } peers { permanent_uuid: "7a4f5f00a15646a3ae17db2711fff3e0" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 34809 } }
I20260806 22:52:22.501428 12211 leader_election.cc:288] T 462fd37583e54d5d97eb8ce70939edf7 P 860a5cffbfbc4199a28144ec07d1fcd3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4233903adc864154b6565c00f5654b70 (127.28.218.195:45531), 7a4f5f00a15646a3ae17db2711fff3e0 (127.28.218.193:34809)
I20260806 22:52:22.504398 12203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:22.501719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.194:40635 (local address 127.28.218.193:34809)
0806 22:52:22.501769 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 22:52:22.501771 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:52:22.501902 (+ 131us) server_negotiation.cc:407] Connection header received
0806 22:52:22.501934 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:52:22.501936 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:52:22.501947 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:52:22.501999 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:52:22.502175 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:22.502650 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:22.503664 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:22.504057 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:22.504140 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:52:22.504149 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 22:52:22.504285 (+ 136us) server_negotiation.cc:299] Negotiation successful
0806 22:52:22.504323 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 22:52:22.504415 12202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:22.501738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.194:55391 (local address 127.28.218.195:45531)
0806 22:52:22.501784 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 22:52:22.501786 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:52:22.501807 (+ 21us) server_negotiation.cc:407] Connection header received
0806 22:52:22.501833 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:52:22.501834 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:52:22.501845 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:52:22.501910 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:52:22.502173 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:22.502651 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:22.503672 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:22.504086 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:22.504169 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:52:22.504177 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 22:52:22.504308 (+ 131us) server_negotiation.cc:299] Negotiation successful
0806 22:52:22.504344 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 22:52:22.504441 12051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:22.501621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.195:45531 (local address 127.28.218.194:55391)
0806 22:52:22.501681 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0806 22:52:22.501695 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 22:52:22.501759 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:52:22.502046 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:52:22.502047 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:52:22.502048 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:52:22.502099 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:52:22.502100 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:52:22.502762 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:52:22.502763 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:52:22.503576 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:52:22.503578 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:52:22.504168 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:52:22.504169 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:52:22.504238 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:52:22.504240 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 22:52:22.504304 (+ 64us) client_negotiation.cc:240] Negotiation successful
0806 22:52:22.504346 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36,"mutex_wait_us":28}
I20260806 22:52:22.504448 12214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:22.501623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.193:34809 (local address 127.28.218.194:40635)
0806 22:52:22.501822 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0806 22:52:22.501831 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:52:22.501893 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:52:22.502021 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:52:22.502022 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:52:22.502023 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:52:22.502090 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:52:22.502091 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:52:22.502755 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:52:22.502756 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:52:22.503575 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:52:22.503577 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:52:22.504138 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:52:22.504139 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:52:22.504210 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:52:22.504214 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:52:22.504280 (+ 66us) client_negotiation.cc:240] Negotiation successful
0806 22:52:22.504320 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"mutex_wait_us":104,"thread_start_us":65,"threads_started":1}
I20260806 22:52:22.504804 12141 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "462fd37583e54d5d97eb8ce70939edf7" candidate_uuid: "860a5cffbfbc4199a28144ec07d1fcd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4233903adc864154b6565c00f5654b70" is_pre_election: true
I20260806 22:52:22.504802 11861 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "462fd37583e54d5d97eb8ce70939edf7" candidate_uuid: "860a5cffbfbc4199a28144ec07d1fcd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a4f5f00a15646a3ae17db2711fff3e0" is_pre_election: true
I20260806 22:52:22.504961 11861 raft_consensus.cc:2454] T 462fd37583e54d5d97eb8ce70939edf7 P 7a4f5f00a15646a3ae17db2711fff3e0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 860a5cffbfbc4199a28144ec07d1fcd3 in term 0.
I20260806 22:52:22.504968 12141 raft_consensus.cc:2454] T 462fd37583e54d5d97eb8ce70939edf7 P 4233903adc864154b6565c00f5654b70 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 860a5cffbfbc4199a28144ec07d1fcd3 in term 0.
I20260806 22:52:22.505213 11933 leader_election.cc:302] T 462fd37583e54d5d97eb8ce70939edf7 P 860a5cffbfbc4199a28144ec07d1fcd3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a4f5f00a15646a3ae17db2711fff3e0, 860a5cffbfbc4199a28144ec07d1fcd3; no voters:
I20260806 22:52:22.505352 12211 raft_consensus.cc:2781] T 462fd37583e54d5d97eb8ce70939edf7 P 860a5cffbfbc4199a28144ec07d1fcd3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:52:22.505424 12211 raft_consensus.cc:482] T 462fd37583e54d5d97eb8ce70939edf7 P 860a5cffbfbc4199a28144ec07d1fcd3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:52:22.505460 12211 raft_consensus.cc:3037] T 462fd37583e54d5d97eb8ce70939edf7 P 860a5cffbfbc4199a28144ec07d1fcd3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:52:22.506258 12211 raft_consensus.cc:504] T 462fd37583e54d5d97eb8ce70939edf7 P 860a5cffbfbc4199a28144ec07d1fcd3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4233903adc864154b6565c00f5654b70" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 45531 } } peers { permanent_uuid: "860a5cffbfbc4199a28144ec07d1fcd3" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 42589 } } peers { permanent_uuid: "7a4f5f00a15646a3ae17db2711fff3e0" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 34809 } }
I20260806 22:52:22.506404 12211 leader_election.cc:288] T 462fd37583e54d5d97eb8ce70939edf7 P 860a5cffbfbc4199a28144ec07d1fcd3 [CANDIDATE]: Term 1 election: Requested vote from peers 4233903adc864154b6565c00f5654b70 (127.28.218.195:45531), 7a4f5f00a15646a3ae17db2711fff3e0 (127.28.218.193:34809)
I20260806 22:52:22.506551 12141 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "462fd37583e54d5d97eb8ce70939edf7" candidate_uuid: "860a5cffbfbc4199a28144ec07d1fcd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4233903adc864154b6565c00f5654b70"
I20260806 22:52:22.506589 11861 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "462fd37583e54d5d97eb8ce70939edf7" candidate_uuid: "860a5cffbfbc4199a28144ec07d1fcd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a4f5f00a15646a3ae17db2711fff3e0"
I20260806 22:52:22.506635 12141 raft_consensus.cc:3037] T 462fd37583e54d5d97eb8ce70939edf7 P 4233903adc864154b6565c00f5654b70 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:52:22.506659 11861 raft_consensus.cc:3037] T 462fd37583e54d5d97eb8ce70939edf7 P 7a4f5f00a15646a3ae17db2711fff3e0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:52:22.507565 12141 raft_consensus.cc:2454] T 462fd37583e54d5d97eb8ce70939edf7 P 4233903adc864154b6565c00f5654b70 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 860a5cffbfbc4199a28144ec07d1fcd3 in term 1.
I20260806 22:52:22.507565 11861 raft_consensus.cc:2454] T 462fd37583e54d5d97eb8ce70939edf7 P 7a4f5f00a15646a3ae17db2711fff3e0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 860a5cffbfbc4199a28144ec07d1fcd3 in term 1.
I20260806 22:52:22.507727 11933 leader_election.cc:302] T 462fd37583e54d5d97eb8ce70939edf7 P 860a5cffbfbc4199a28144ec07d1fcd3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a4f5f00a15646a3ae17db2711fff3e0, 860a5cffbfbc4199a28144ec07d1fcd3; no voters:
I20260806 22:52:22.507850 12211 raft_consensus.cc:2781] T 462fd37583e54d5d97eb8ce70939edf7 P 860a5cffbfbc4199a28144ec07d1fcd3 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:52:22.507990 12211 raft_consensus.cc:686] T 462fd37583e54d5d97eb8ce70939edf7 P 860a5cffbfbc4199a28144ec07d1fcd3 [term 1 LEADER]: Becoming Leader. State: Replica: 860a5cffbfbc4199a28144ec07d1fcd3, State: Running, Role: LEADER
I20260806 22:52:22.508100 12211 consensus_queue.cc:237] T 462fd37583e54d5d97eb8ce70939edf7 P 860a5cffbfbc4199a28144ec07d1fcd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4233903adc864154b6565c00f5654b70" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 45531 } } peers { permanent_uuid: "860a5cffbfbc4199a28144ec07d1fcd3" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 42589 } } peers { permanent_uuid: "7a4f5f00a15646a3ae17db2711fff3e0" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 34809 } }
I20260806 22:52:22.508958 11708 catalog_manager.cc:5495] T 462fd37583e54d5d97eb8ce70939edf7 P 860a5cffbfbc4199a28144ec07d1fcd3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 860a5cffbfbc4199a28144ec07d1fcd3 (127.28.218.194). New cstate: current_term: 1 leader_uuid: "860a5cffbfbc4199a28144ec07d1fcd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4233903adc864154b6565c00f5654b70" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 45531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "860a5cffbfbc4199a28144ec07d1fcd3" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 42589 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a4f5f00a15646a3ae17db2711fff3e0" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 34809 } health_report { overall_health: UNKNOWN } } }
I20260806 22:52:22.537385 12204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:22.535017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59118 (local address 127.28.218.194:42589)
0806 22:52:22.535086 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0806 22:52:22.535088 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:52:22.535097 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:52:22.535171 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:52:22.535172 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:52:22.535195 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:52:22.535287 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:52:22.535420 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:22.535871 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:22.536612 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:22.536797 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:22.536876 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:52:22.537035 (+ 159us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:52:22.537134 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:52:22.537236 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0806 22:52:22.537256 (+ 20us) server_negotiation.cc:299] Negotiation successful
0806 22:52:22.537275 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 22:52:22.539192 12141 raft_consensus.cc:1261] T 462fd37583e54d5d97eb8ce70939edf7 P 4233903adc864154b6565c00f5654b70 [term 1 FOLLOWER]: Refusing update from remote peer 860a5cffbfbc4199a28144ec07d1fcd3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:52:22.539232 11861 raft_consensus.cc:1261] T 462fd37583e54d5d97eb8ce70939edf7 P 7a4f5f00a15646a3ae17db2711fff3e0 [term 1 FOLLOWER]: Refusing update from remote peer 860a5cffbfbc4199a28144ec07d1fcd3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:52:22.539523 12211 consensus_queue.cc:1035] T 462fd37583e54d5d97eb8ce70939edf7 P 860a5cffbfbc4199a28144ec07d1fcd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4233903adc864154b6565c00f5654b70" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 45531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:52:22.539646 12215 consensus_queue.cc:1035] T 462fd37583e54d5d97eb8ce70939edf7 P 860a5cffbfbc4199a28144ec07d1fcd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a4f5f00a15646a3ae17db2711fff3e0" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 34809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:52:22.541810 12219 mvcc.cc:205] Tried to move back new op lower bound from 7315688417437831168 to 7315688417313832960. Current Snapshot: MvccSnapshot[applied={T|T < 7315688417437831168}]
I20260806 22:52:22.542034 12221 mvcc.cc:205] Tried to move back new op lower bound from 7315688417437831168 to 7315688417313832960. Current Snapshot: MvccSnapshot[applied={T|T < 7315688417437831168}]
I20260806 22:52:22.542897 12220 mvcc.cc:205] Tried to move back new op lower bound from 7315688417437831168 to 7315688417313832960. Current Snapshot: MvccSnapshot[applied={T|T < 7315688417437831168 or (T in {7315688417437831168})}]
I20260806 22:52:22.544276 11708 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38314:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:52:22.544371 11708 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38314:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:52:22.545126 11708 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d312610ebc6a4ab3aaec82261c28aa09: Sending DeleteTablet for 3 replicas of tablet 462fd37583e54d5d97eb8ce70939edf7
I20260806 22:52:22.545475 12121 tablet_service.cc:1514] Processing DeleteTablet for tablet 462fd37583e54d5d97eb8ce70939edf7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:52:22 UTC) from {username='kudu', principal='kudu/127.28.218.254@KRBTEST.COM'} at 127.0.0.1:54902
I20260806 22:52:22.545485 11978 tablet_service.cc:1514] Processing DeleteTablet for tablet 462fd37583e54d5d97eb8ce70939edf7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:52:22 UTC) from {username='kudu', principal='kudu/127.28.218.254@KRBTEST.COM'} at 127.0.0.1:59110
I20260806 22:52:22.545710 11839 tablet_service.cc:1514] Processing DeleteTablet for tablet 462fd37583e54d5d97eb8ce70939edf7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:52:22 UTC) from {username='kudu', principal='kudu/127.28.218.254@KRBTEST.COM'} at 127.0.0.1:47114
I20260806 22:52:22.545763 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 11774
I20260806 22:52:22.545770 12232 tablet_replica.cc:330] T 462fd37583e54d5d97eb8ce70939edf7 P 4233903adc864154b6565c00f5654b70: stopping tablet replica
I20260806 22:52:22.545861 12232 raft_consensus.cc:2229] T 462fd37583e54d5d97eb8ce70939edf7 P 4233903adc864154b6565c00f5654b70 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:52:22.545946 12232 raft_consensus.cc:2258] T 462fd37583e54d5d97eb8ce70939edf7 P 4233903adc864154b6565c00f5654b70 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:52:22.546088 12230 tablet_replica.cc:330] T 462fd37583e54d5d97eb8ce70939edf7 P 860a5cffbfbc4199a28144ec07d1fcd3: stopping tablet replica
I20260806 22:52:22.546171 12230 raft_consensus.cc:2229] T 462fd37583e54d5d97eb8ce70939edf7 P 860a5cffbfbc4199a28144ec07d1fcd3 [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:52:22.546587 12230 raft_consensus.cc:2258] T 462fd37583e54d5d97eb8ce70939edf7 P 860a5cffbfbc4199a28144ec07d1fcd3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:52:22.547080 12230 ts_tablet_manager.cc:1905] T 462fd37583e54d5d97eb8ce70939edf7 P 860a5cffbfbc4199a28144ec07d1fcd3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:52:22.547761 12232 ts_tablet_manager.cc:1905] T 462fd37583e54d5d97eb8ce70939edf7 P 4233903adc864154b6565c00f5654b70: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:52:22.549080 12230 ts_tablet_manager.cc:1918] T 462fd37583e54d5d97eb8ce70939edf7 P 860a5cffbfbc4199a28144ec07d1fcd3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:52:22.549154 12230 log.cc:1196] T 462fd37583e54d5d97eb8ce70939edf7 P 860a5cffbfbc4199a28144ec07d1fcd3: Deleting WAL directory at /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-1/wal/wals/462fd37583e54d5d97eb8ce70939edf7
I20260806 22:52:22.549414 12230 ts_tablet_manager.cc:1939] T 462fd37583e54d5d97eb8ce70939edf7 P 860a5cffbfbc4199a28144ec07d1fcd3: Deleting consensus metadata
I20260806 22:52:22.549721 12232 ts_tablet_manager.cc:1918] T 462fd37583e54d5d97eb8ce70939edf7 P 4233903adc864154b6565c00f5654b70: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:52:22.549801 12232 log.cc:1196] T 462fd37583e54d5d97eb8ce70939edf7 P 4233903adc864154b6565c00f5654b70: Deleting WAL directory at /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-2/wal/wals/462fd37583e54d5d97eb8ce70939edf7
I20260806 22:52:22.549809 11690 catalog_manager.cc:4841] TS 860a5cffbfbc4199a28144ec07d1fcd3 (127.28.218.194:42589): tablet 462fd37583e54d5d97eb8ce70939edf7 (table test-table [id=b0ee7b49c0e743dc9751b653532dd0d0]) successfully deleted
I20260806 22:52:22.550074 12232 ts_tablet_manager.cc:1939] T 462fd37583e54d5d97eb8ce70939edf7 P 4233903adc864154b6565c00f5654b70: Deleting consensus metadata
I20260806 22:52:22.550400 11690 catalog_manager.cc:4841] TS 4233903adc864154b6565c00f5654b70 (127.28.218.195:45531): tablet 462fd37583e54d5d97eb8ce70939edf7 (table test-table [id=b0ee7b49c0e743dc9751b653532dd0d0]) successfully deleted
W20260806 22:52:22.553563 11694 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.28.218.193:34809 (error 108)
I20260806 22:52:22.553750 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 11913
I20260806 22:52:22.555408 12199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:22.555256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.193:34809 (local address 127.0.0.1:47128)
0806 22:52:22.555319 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0806 22:52:22.555368 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.193:34809: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260806 22:52:22.555532 11694 catalog_manager.cc:4584] TS 7a4f5f00a15646a3ae17db2711fff3e0 (127.28.218.193:34809): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 462fd37583e54d5d97eb8ce70939edf7: Network error: Client connection negotiation failed: client connection to 127.28.218.193:34809: connect: Connection refused (error 111)
I20260806 22:52:22.559237 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 12054
I20260806 22:52:22.565513 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 11674
I20260806 22:52:22.572353 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.218.254:45957
--webserver_interface=127.28.218.254
--webserver_port=33089
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab
--principal=kudu/127.28.218.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.218.212:41905
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.218.254:45957
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:52:22.652495 12234 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:52:22.652782 12234 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:52:22.652842 12234 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:52:22.653926 12234 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:52:22.653999 12234 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:52:22.654024 12234 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:52:22.654053 12234 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:52:22.654083 12234 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:52:22.655534 12234 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:41905
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.28.218.254:45957
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab
--principal=kudu/127.28.218.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.218.254:45957
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.218.254
--webserver_port=33089
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12234
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:52:22.655794 12234 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:52:22.656019 12234 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:52:22.664736 12243 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:52:22.664749 12241 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:52:22.664788 12240 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:52:22.664973 12234 server_base.cc:1034] running on GCE node
I20260806 22:52:22.665187 12234 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:52:22.665436 12234 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:52:22.666574 12234 hybrid_clock.cc:648] HybridClock initialized: now 1786056742666547 us; error 44 us; skew 500 ppm
Aug 06 22:52:22 dist-test-slave-1xrh krb5kdc[11659](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056742, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:52:22.674850 12234 init.cc:377] Logged in from keytab as kudu/127.28.218.254@KRBTEST.COM (short username kudu)
I20260806 22:52:22.675359 12234 webserver.cc:460] Webserver started at http://127.28.218.254:33089/ using document root <none> and password file <none>
I20260806 22:52:22.675567 12234 fs_manager.cc:359] Metadata directory not provided
I20260806 22:52:22.675621 12234 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:52:22.676867 12234 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:52:22.677516 12251 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:52:22.677690 12234 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:52:22.677798 12234 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/master-0/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/master-0/wal
uuid: "d312610ebc6a4ab3aaec82261c28aa09"
format_stamp: "Formatted at 2026-08-06 22:52:21 on dist-test-slave-1xrh"
server_key: "299f120946a4497f663707ffd8d20458"
server_key_iv: "375f6ffbd4a558e198668ceaa76da574"
server_key_version: "encryptionkey@0"
I20260806 22:52:22.678161 12234 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:52:22.692348 12234 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:52:22.694540 12234 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:52:22.694691 12234 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:52:22.698889 12234 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.254:45957
I20260806 22:52:22.698925 12308 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.254:45957 every 8 connection(s)
I20260806 22:52:22.699349 12234 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
I20260806 22:52:22.700999 12309 sys_catalog.cc:263] Verifying existing consensus state
I20260806 22:52:22.701661 12309 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d312610ebc6a4ab3aaec82261c28aa09: Bootstrap starting.
I20260806 22:52:22.707360 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 12234
I20260806 22:52:22.707706 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.218.193:34809
--local_ip_for_outbound_sockets=127.28.218.193
--tserver_master_addrs=127.28.218.254:45957
--webserver_port=44957
--webserver_interface=127.28.218.193
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.193.keytab
--principal=kudu/127.28.218.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.218.212:41905
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:52:22.710605 12309 log.cc:824] T 00000000000000000000000000000000 P d312610ebc6a4ab3aaec82261c28aa09: Log is configured to *not* fsync() on all Append() calls
I20260806 22:52:22.712493 12309 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d312610ebc6a4ab3aaec82261c28aa09: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 22:52:22.712798 12309 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d312610ebc6a4ab3aaec82261c28aa09: Bootstrap complete.
I20260806 22:52:22.714874 12309 raft_consensus.cc:348] T 00000000000000000000000000000000 P d312610ebc6a4ab3aaec82261c28aa09 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d312610ebc6a4ab3aaec82261c28aa09" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 45957 } }
I20260806 22:52:22.715067 12309 raft_consensus.cc:729] T 00000000000000000000000000000000 P d312610ebc6a4ab3aaec82261c28aa09 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d312610ebc6a4ab3aaec82261c28aa09, State: Initialized, Role: FOLLOWER
I20260806 22:52:22.715227 12309 consensus_queue.cc:260] T 00000000000000000000000000000000 P d312610ebc6a4ab3aaec82261c28aa09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d312610ebc6a4ab3aaec82261c28aa09" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 45957 } }
I20260806 22:52:22.715318 12309 raft_consensus.cc:388] T 00000000000000000000000000000000 P d312610ebc6a4ab3aaec82261c28aa09 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:52:22.715386 12309 raft_consensus.cc:482] T 00000000000000000000000000000000 P d312610ebc6a4ab3aaec82261c28aa09 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:52:22.715442 12309 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d312610ebc6a4ab3aaec82261c28aa09 [term 1 FOLLOWER]: Advancing to term 2
I20260806 22:52:22.716410 12309 raft_consensus.cc:504] T 00000000000000000000000000000000 P d312610ebc6a4ab3aaec82261c28aa09 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d312610ebc6a4ab3aaec82261c28aa09" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 45957 } }
I20260806 22:52:22.716506 12309 leader_election.cc:302] T 00000000000000000000000000000000 P d312610ebc6a4ab3aaec82261c28aa09 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d312610ebc6a4ab3aaec82261c28aa09; no voters:
I20260806 22:52:22.716691 12309 leader_election.cc:288] T 00000000000000000000000000000000 P d312610ebc6a4ab3aaec82261c28aa09 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 22:52:22.716727 12315 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d312610ebc6a4ab3aaec82261c28aa09 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 22:52:22.716828 12315 raft_consensus.cc:686] T 00000000000000000000000000000000 P d312610ebc6a4ab3aaec82261c28aa09 [term 2 LEADER]: Becoming Leader. State: Replica: d312610ebc6a4ab3aaec82261c28aa09, State: Running, Role: LEADER
I20260806 22:52:22.716940 12309 sys_catalog.cc:564] T 00000000000000000000000000000000 P d312610ebc6a4ab3aaec82261c28aa09 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:52:22.716917 12315 consensus_queue.cc:237] T 00000000000000000000000000000000 P d312610ebc6a4ab3aaec82261c28aa09 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d312610ebc6a4ab3aaec82261c28aa09" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 45957 } }
I20260806 22:52:22.717237 12315 sys_catalog.cc:455] T 00000000000000000000000000000000 P d312610ebc6a4ab3aaec82261c28aa09 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d312610ebc6a4ab3aaec82261c28aa09. Latest consensus state: current_term: 2 leader_uuid: "d312610ebc6a4ab3aaec82261c28aa09" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d312610ebc6a4ab3aaec82261c28aa09" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 45957 } } }
I20260806 22:52:22.717339 12315 sys_catalog.cc:458] T 00000000000000000000000000000000 P d312610ebc6a4ab3aaec82261c28aa09 [sys.catalog]: This master's current role is: LEADER
I20260806 22:52:22.717247 12316 sys_catalog.cc:455] T 00000000000000000000000000000000 P d312610ebc6a4ab3aaec82261c28aa09 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "d312610ebc6a4ab3aaec82261c28aa09" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d312610ebc6a4ab3aaec82261c28aa09" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 45957 } } }
I20260806 22:52:22.717532 12316 sys_catalog.cc:458] T 00000000000000000000000000000000 P d312610ebc6a4ab3aaec82261c28aa09 [sys.catalog]: This master's current role is: LEADER
I20260806 22:52:22.717541 12320 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:52:22.718474 12320 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:52:22.718617 12320 catalog_manager.cc:1259] Loaded cluster ID: 8c5f4590e8804d2dbfed5c3f208eb9e0
I20260806 22:52:22.718850 12320 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:52:22.719168 12320 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260806 22:52:22.724279 12320 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260806 22:52:22.726620 12320 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:52:22.726810 12320 catalog_manager.cc:5879] T 00000000000000000000000000000000 P d312610ebc6a4ab3aaec82261c28aa09: Loaded TSK: 0
I20260806 22:52:22.727038 12320 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 22:52:22.791442 12313 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:52:22.791683 12313 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:52:22.791719 12313 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:52:22.792793 12313 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:52:22.792857 12313 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:52:22.792899 12313 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.218.193
I20260806 22:52:22.794234 12313 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:41905
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.193.keytab
--principal=kudu/127.28.218.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.218.193:34809
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.218.193
--webserver_port=44957
--tserver_master_addrs=127.28.218.254:45957
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12313
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.218.193
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:52:22.794510 12313 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:52:22.794723 12313 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:52:22.799734 12340 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:52:22.799738 12338 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:52:22.799754 12342 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:52:22.800336 12313 server_base.cc:1034] running on GCE node
I20260806 22:52:22.800549 12313 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:52:22.800809 12313 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:52:22.801961 12313 hybrid_clock.cc:648] HybridClock initialized: now 1786056742801950 us; error 27 us; skew 500 ppm
Aug 06 22:52:22 dist-test-slave-1xrh krb5kdc[11659](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056742, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:52:22.804466 12313 init.cc:377] Logged in from keytab as kudu/127.28.218.193@KRBTEST.COM (short username kudu)
I20260806 22:52:22.804922 12313 webserver.cc:460] Webserver started at http://127.28.218.193:44957/ using document root <none> and password file <none>
I20260806 22:52:22.805146 12313 fs_manager.cc:359] Metadata directory not provided
I20260806 22:52:22.805205 12313 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:52:22.806542 12313 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:52:22.807251 12349 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:52:22.807389 12313 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:52:22.807492 12313 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-0/wal
uuid: "7a4f5f00a15646a3ae17db2711fff3e0"
format_stamp: "Formatted at 2026-08-06 22:52:21 on dist-test-slave-1xrh"
server_key: "649619ee3e06e37c564b970f1879a0a3"
server_key_iv: "d920b093ce9032370d9745a810cb0925"
server_key_version: "encryptionkey@0"
I20260806 22:52:22.807791 12313 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:52:22.830001 12313 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:52:22.832202 12313 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:52:22.832343 12313 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:52:22.832604 12313 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:52:22.833132 12356 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 22:52:22.834470 12313 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 22:52:22.834517 12313 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260806 22:52:22.834550 12313 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 22:52:22.835227 12313 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 22:52:22.835278 12313 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.001s sys 0.000s
I20260806 22:52:22.835333 12356 tablet_bootstrap.cc:492] T 462fd37583e54d5d97eb8ce70939edf7 P 7a4f5f00a15646a3ae17db2711fff3e0: Bootstrap starting.
I20260806 22:52:22.843070 12313 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.193:34809
I20260806 22:52:22.843186 12468 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.193:34809 every 8 connection(s)
I20260806 22:52:22.843586 12313 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-0/data/info.pb
I20260806 22:52:22.846253 12356 log.cc:824] T 462fd37583e54d5d97eb8ce70939edf7 P 7a4f5f00a15646a3ae17db2711fff3e0: Log is configured to *not* fsync() on all Append() calls
I20260806 22:52:22.848218 12356 tablet_bootstrap.cc:492] T 462fd37583e54d5d97eb8ce70939edf7 P 7a4f5f00a15646a3ae17db2711fff3e0: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 22:52:22.848503 12356 tablet_bootstrap.cc:492] T 462fd37583e54d5d97eb8ce70939edf7 P 7a4f5f00a15646a3ae17db2711fff3e0: Bootstrap complete.
I20260806 22:52:22.849392 12356 ts_tablet_manager.cc:1397] T 462fd37583e54d5d97eb8ce70939edf7 P 7a4f5f00a15646a3ae17db2711fff3e0: Time spent bootstrapping tablet: real 0.014s user 0.001s sys 0.011s
I20260806 22:52:22.850313 12356 raft_consensus.cc:348] T 462fd37583e54d5d97eb8ce70939edf7 P 7a4f5f00a15646a3ae17db2711fff3e0 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4233903adc864154b6565c00f5654b70" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 45531 } } peers { permanent_uuid: "860a5cffbfbc4199a28144ec07d1fcd3" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 42589 } } peers { permanent_uuid: "7a4f5f00a15646a3ae17db2711fff3e0" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 34809 } }
I20260806 22:52:22.850555 12356 raft_consensus.cc:729] T 462fd37583e54d5d97eb8ce70939edf7 P 7a4f5f00a15646a3ae17db2711fff3e0 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a4f5f00a15646a3ae17db2711fff3e0, State: Initialized, Role: FOLLOWER
I20260806 22:52:22.850693 12356 consensus_queue.cc:260] T 462fd37583e54d5d97eb8ce70939edf7 P 7a4f5f00a15646a3ae17db2711fff3e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4233903adc864154b6565c00f5654b70" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 45531 } } peers { permanent_uuid: "860a5cffbfbc4199a28144ec07d1fcd3" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 42589 } } peers { permanent_uuid: "7a4f5f00a15646a3ae17db2711fff3e0" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 34809 } }
I20260806 22:52:22.851007 12356 ts_tablet_manager.cc:1428] T 462fd37583e54d5d97eb8ce70939edf7 P 7a4f5f00a15646a3ae17db2711fff3e0: Time spent starting tablet: real 0.002s user 0.004s sys 0.000s
Aug 06 22:52:22 dist-test-slave-1xrh krb5kdc[11659](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056742, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.193@KRBTEST.COM for kudu/127.28.218.254@KRBTEST.COM
I20260806 22:52:22.853816 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 12313
I20260806 22:52:22.854118 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.218.194:42589
--local_ip_for_outbound_sockets=127.28.218.194
--tserver_master_addrs=127.28.218.254:45957
--webserver_port=36183
--webserver_interface=127.28.218.194
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.194.keytab
--principal=kudu/127.28.218.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.218.212:41905
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:52:22.856053 12471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:22.844277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:45957 (local address 127.28.218.193:38049)
0806 22:52:22.844467 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0806 22:52:22.844509 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 22:52:22.844947 (+ 438us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:52:22.846854 (+ 1907us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:52:22.846858 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:52:22.847207 (+ 349us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:52:22.847387 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:52:22.847392 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:52:22.848495 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:52:22.848496 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:52:22.849312 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:52:22.849315 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:52:22.849972 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:52:22.849974 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:52:22.850113 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:52:22.850718 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:52:22.850735 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:52:22.852241 (+ 1506us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:52:22.854344 (+ 2103us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:52:22.854349 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:52:22.854351 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:52:22.854551 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:52:22.854885 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:52:22.854886 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:52:22.854886 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:52:22.855005 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:52:22.855460 (+ 455us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:52:22.855465 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:52:22.855570 (+ 105us) client_negotiation.cc:769] Sending connection context
0806 22:52:22.855655 (+ 85us) client_negotiation.cc:240] Negotiation successful
0806 22:52:22.855806 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":112,"threads_started":1}
I20260806 22:52:22.856069 12472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:22.844393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.193:38049 (local address 127.28.218.254:45957)
0806 22:52:22.846508 (+ 2115us) server_negotiation.cc:206] Beginning negotiation
0806 22:52:22.846515 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:52:22.846541 (+ 26us) server_negotiation.cc:407] Connection header received
0806 22:52:22.846641 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:52:22.846645 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:52:22.846684 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:52:22.846822 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:52:22.847520 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:22.848388 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:22.849415 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:22.849862 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:22.849981 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:52:22.852342 (+ 2361us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:52:22.852346 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:52:22.852346 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:52:22.852356 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:52:22.854252 (+ 1896us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:52:22.854652 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:52:22.854655 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:52:22.854656 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:52:22.854783 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:52:22.855080 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:52:22.855081 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:52:22.855081 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:52:22.855388 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:52:22.855475 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 22:52:22.855731 (+ 256us) server_negotiation.cc:299] Negotiation successful
0806 22:52:22.855888 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2051,"thread_start_us":93,"threads_started":1}
I20260806 22:52:22.856814 12469 heartbeater.cc:347] Connected to a master server at 127.28.218.254:45957
I20260806 22:52:22.856902 12469 heartbeater.cc:464] Registering TS with master...
I20260806 22:52:22.857069 12469 heartbeater.cc:511] Master 127.28.218.254:45957 requested a full tablet report, sending...
I20260806 22:52:22.857738 12269 ts_manager.cc:194] Registered new tserver with Master: 7a4f5f00a15646a3ae17db2711fff3e0 (127.28.218.193:34809)
I20260806 22:52:22.858726 12269 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.218.193@KRBTEST.COM'} at 127.28.218.193:38049
I20260806 22:52:22.859740 12469 heartbeater.cc:503] Master 127.28.218.254:45957 was elected leader, sending a full tablet report...
Aug 06 22:52:22 dist-test-slave-1xrh krb5kdc[11659](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056742, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.254@KRBTEST.COM for kudu/127.28.218.193@KRBTEST.COM
I20260806 22:52:22.866773 12477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:22.858415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47142 (local address 127.28.218.193:34809)
0806 22:52:22.859267 (+ 852us) server_negotiation.cc:206] Beginning negotiation
0806 22:52:22.859270 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:52:22.859277 (+ 7us) server_negotiation.cc:407] Connection header received
0806 22:52:22.859315 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:52:22.859317 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:52:22.859328 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:52:22.859409 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:52:22.859773 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:22.860672 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:22.861407 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:22.861692 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:22.861778 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:52:22.863964 (+ 2186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:52:22.863972 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:52:22.863972 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:52:22.863979 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:52:22.865518 (+ 1539us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:52:22.865918 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:52:22.865920 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:52:22.865921 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:52:22.865979 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:52:22.866192 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:52:22.866193 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:52:22.866193 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:52:22.866322 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:52:22.866414 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0806 22:52:22.866608 (+ 194us) server_negotiation.cc:299] Negotiation successful
0806 22:52:22.866690 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":815,"thread_start_us":73,"threads_started":1}
I20260806 22:52:22.866783 12476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:22.858347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.193:34809 (local address 127.0.0.1:47142)
0806 22:52:22.858998 (+ 651us) negotiation.cc:107] Waiting for socket to connect
0806 22:52:22.859039 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 22:52:22.859107 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:52:22.859424 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:52:22.859428 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:52:22.859659 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:52:22.859696 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:52:22.859697 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:52:22.860756 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:52:22.860757 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:52:22.861323 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:52:22.861325 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:52:22.861783 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:52:22.861784 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:52:22.861857 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:52:22.862303 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:52:22.862308 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:52:22.863839 (+ 1531us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:52:22.865628 (+ 1789us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:52:22.865635 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:52:22.865636 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:52:22.865833 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:52:22.866060 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:52:22.866061 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:52:22.866061 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:52:22.866124 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:52:22.866396 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:52:22.866402 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:52:22.866464 (+ 62us) client_negotiation.cc:769] Sending connection context
0806 22:52:22.866561 (+ 97us) client_negotiation.cc:240] Negotiation successful
0806 22:52:22.866676 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":632,"thread_start_us":83,"threads_started":1}
I20260806 22:52:22.867384 12399 tablet_service.cc:1514] Processing DeleteTablet for tablet 462fd37583e54d5d97eb8ce70939edf7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:52:22 UTC) from {username='kudu', principal='kudu/127.28.218.254@KRBTEST.COM'} at 127.0.0.1:47142
I20260806 22:52:22.867802 12478 tablet_replica.cc:330] stopping tablet replica
I20260806 22:52:22.867906 12478 raft_consensus.cc:2229] T 462fd37583e54d5d97eb8ce70939edf7 P 7a4f5f00a15646a3ae17db2711fff3e0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:52:22.867964 12478 raft_consensus.cc:2258] T 462fd37583e54d5d97eb8ce70939edf7 P 7a4f5f00a15646a3ae17db2711fff3e0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:52:22.868229 12478 ts_tablet_manager.cc:1905] T 462fd37583e54d5d97eb8ce70939edf7 P 7a4f5f00a15646a3ae17db2711fff3e0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:52:22.870253 12478 ts_tablet_manager.cc:1918] T 462fd37583e54d5d97eb8ce70939edf7 P 7a4f5f00a15646a3ae17db2711fff3e0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:52:22.870314 12478 log.cc:1196] T 462fd37583e54d5d97eb8ce70939edf7 P 7a4f5f00a15646a3ae17db2711fff3e0: Deleting WAL directory at /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-0/wal/wals/462fd37583e54d5d97eb8ce70939edf7
I20260806 22:52:22.870530 12478 ts_tablet_manager.cc:1939] T 462fd37583e54d5d97eb8ce70939edf7 P 7a4f5f00a15646a3ae17db2711fff3e0: Deleting consensus metadata
I20260806 22:52:22.870971 12255 catalog_manager.cc:4841] TS 7a4f5f00a15646a3ae17db2711fff3e0 (127.28.218.193:34809): tablet 462fd37583e54d5d97eb8ce70939edf7 (table test-table [id=b0ee7b49c0e743dc9751b653532dd0d0]) successfully deleted
W20260806 22:52:22.936254 12475 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:52:22.936480 12475 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:52:22.936522 12475 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:52:22.937639 12475 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:52:22.937703 12475 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:52:22.937747 12475 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.218.194
I20260806 22:52:22.939213 12475 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:41905
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.194.keytab
--principal=kudu/127.28.218.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.218.194:42589
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.218.194
--webserver_port=36183
--tserver_master_addrs=127.28.218.254:45957
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12475
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.218.194
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:52:22.939484 12475 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:52:22.939702 12475 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:52:22.944806 12484 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:52:22.944828 12486 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:52:22.944842 12483 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:52:22.945122 12475 server_base.cc:1034] running on GCE node
I20260806 22:52:22.945305 12475 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:52:22.945535 12475 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:52:22.946709 12475 hybrid_clock.cc:648] HybridClock initialized: now 1786056742946696 us; error 26 us; skew 500 ppm
Aug 06 22:52:22 dist-test-slave-1xrh krb5kdc[11659](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056742, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:52:22.949029 12475 init.cc:377] Logged in from keytab as kudu/127.28.218.194@KRBTEST.COM (short username kudu)
I20260806 22:52:22.949440 12475 webserver.cc:460] Webserver started at http://127.28.218.194:36183/ using document root <none> and password file <none>
I20260806 22:52:22.949615 12475 fs_manager.cc:359] Metadata directory not provided
I20260806 22:52:22.949663 12475 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:52:22.950826 12475 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:52:22.951448 12493 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:52:22.951603 12475 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:52:22.951683 12475 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-1/wal
uuid: "860a5cffbfbc4199a28144ec07d1fcd3"
format_stamp: "Formatted at 2026-08-06 22:52:21 on dist-test-slave-1xrh"
server_key: "ea7cf74de7358c4022b4b16d2b569640"
server_key_iv: "c45dfeff539ec2dd568e793ed682e328"
server_key_version: "encryptionkey@0"
I20260806 22:52:22.951960 12475 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:52:22.965602 12475 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:52:22.967831 12475 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:52:22.967934 12475 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:52:22.968093 12475 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:52:22.968387 12475 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:52:22.968421 12475 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:52:22.968441 12475 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:52:22.968452 12475 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:52:22.974093 12475 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.194:42589
I20260806 22:52:22.974164 12612 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.194:42589 every 8 connection(s)
I20260806 22:52:22.974474 12475 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-1/data/info.pb
I20260806 22:52:22.980090 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 12475
Aug 06 22:52:22 dist-test-slave-1xrh krb5kdc[11659](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056742, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.194@KRBTEST.COM for kudu/127.28.218.254@KRBTEST.COM
I20260806 22:52:22.980437 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.218.195:45531
--local_ip_for_outbound_sockets=127.28.218.195
--tserver_master_addrs=127.28.218.254:45957
--webserver_port=41411
--webserver_interface=127.28.218.195
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.195.keytab
--principal=kudu/127.28.218.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.218.212:41905
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:52:22.983817 12615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:22.975117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:45957 (local address 127.28.218.194:56775)
0806 22:52:22.975283 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0806 22:52:22.975368 (+ 85us) client_negotiation.cc:174] Beginning negotiation
0806 22:52:22.975839 (+ 471us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:52:22.976082 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:52:22.976086 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:52:22.976638 (+ 552us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:52:22.976803 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:52:22.976807 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:52:22.977510 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:52:22.977512 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:52:22.978262 (+ 750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:52:22.978263 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:52:22.978673 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:52:22.978674 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:52:22.978791 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:52:22.979226 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:52:22.979234 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:52:22.980738 (+ 1504us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:52:22.982490 (+ 1752us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:52:22.982496 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:52:22.982497 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:52:22.982704 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:52:22.982906 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:52:22.982907 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:52:22.982907 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:52:22.982997 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:52:22.983312 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:52:22.983317 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:52:22.983449 (+ 132us) client_negotiation.cc:769] Sending connection context
0806 22:52:22.983537 (+ 88us) client_negotiation.cc:240] Negotiation successful
0806 22:52:22.983651 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":78,"threads_started":1}
I20260806 22:52:22.983827 12472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:22.975171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.194:56775 (local address 127.28.218.254:45957)
0806 22:52:22.975218 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 22:52:22.975220 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:52:22.975445 (+ 225us) server_negotiation.cc:407] Connection header received
0806 22:52:22.975946 (+ 501us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:52:22.975947 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:52:22.975963 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:52:22.976017 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:52:22.976903 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:22.977394 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:22.978354 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:22.978577 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:22.978664 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:52:22.980834 (+ 2170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:52:22.980836 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:52:22.980836 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:52:22.980845 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:52:22.982401 (+ 1556us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:52:22.982778 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:52:22.982779 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:52:22.982780 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:52:22.982842 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:52:22.983061 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:52:22.983062 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:52:22.983063 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:52:22.983240 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:52:22.983317 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 22:52:22.983581 (+ 264us) server_negotiation.cc:299] Negotiation successful
0806 22:52:22.983678 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 22:52:22.984267 12613 heartbeater.cc:347] Connected to a master server at 127.28.218.254:45957
I20260806 22:52:22.984359 12613 heartbeater.cc:464] Registering TS with master...
I20260806 22:52:22.984503 12613 heartbeater.cc:511] Master 127.28.218.254:45957 requested a full tablet report, sending...
I20260806 22:52:22.984827 12269 ts_manager.cc:194] Registered new tserver with Master: 860a5cffbfbc4199a28144ec07d1fcd3 (127.28.218.194:42589)
I20260806 22:52:22.985415 12269 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.218.194@KRBTEST.COM'} at 127.28.218.194:56775
W20260806 22:52:23.062655 12616 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:52:23.062824 12616 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:52:23.062844 12616 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:52:23.063938 12616 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:52:23.063984 12616 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:52:23.064009 12616 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.218.195
I20260806 22:52:23.065366 12616 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:41905
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.195.keytab
--principal=kudu/127.28.218.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.218.195:45531
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.218.195
--webserver_port=41411
--tserver_master_addrs=127.28.218.254:45957
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12616
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.218.195
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:52:23.065593 12616 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:52:23.065809 12616 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:52:23.070590 12624 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:52:23.070618 12621 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:52:23.070560 12622 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:52:23.070765 12616 server_base.cc:1034] running on GCE node
I20260806 22:52:23.070955 12616 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:52:23.071237 12616 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:52:23.072388 12616 hybrid_clock.cc:648] HybridClock initialized: now 1786056743072375 us; error 23 us; skew 500 ppm
Aug 06 22:52:23 dist-test-slave-1xrh krb5kdc[11659](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056743, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:52:23.074883 12616 init.cc:377] Logged in from keytab as kudu/127.28.218.195@KRBTEST.COM (short username kudu)
I20260806 22:52:23.075356 12616 webserver.cc:460] Webserver started at http://127.28.218.195:41411/ using document root <none> and password file <none>
I20260806 22:52:23.075600 12616 fs_manager.cc:359] Metadata directory not provided
I20260806 22:52:23.075678 12616 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:52:23.076989 12616 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:52:23.077569 12633 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:52:23.077726 12616 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:52:23.077821 12616 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-2/wal
uuid: "4233903adc864154b6565c00f5654b70"
format_stamp: "Formatted at 2026-08-06 22:52:22 on dist-test-slave-1xrh"
server_key: "cf2c4923bd749fe655cf5a8c0ad4a6c6"
server_key_iv: "e4ba17445af6dc4914b6655b98d54357"
server_key_version: "encryptionkey@0"
I20260806 22:52:23.078143 12616 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:52:23.098272 12616 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:52:23.100724 12616 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:52:23.100896 12616 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:52:23.101128 12616 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:52:23.101428 12616 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:52:23.101481 12616 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:52:23.101518 12616 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:52:23.101548 12616 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:52:23.107543 12616 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.195:45531
I20260806 22:52:23.107579 12751 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.195:45531 every 8 connection(s)
I20260806 22:52:23.107908 12616 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-2/data/info.pb
Aug 06 22:52:23 dist-test-slave-1xrh krb5kdc[11659](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056743, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.195@KRBTEST.COM for kudu/127.28.218.254@KRBTEST.COM
I20260806 22:52:23.117244 12472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:23.108602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.195:52107 (local address 127.28.218.254:45957)
0806 22:52:23.108679 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0806 22:52:23.108682 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:52:23.108858 (+ 176us) server_negotiation.cc:407] Connection header received
0806 22:52:23.109438 (+ 580us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:52:23.109440 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:52:23.109453 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:52:23.109537 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:52:23.110522 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:23.111064 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:23.112136 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:23.112339 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:23.112414 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:52:23.114321 (+ 1907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:52:23.114323 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:52:23.114323 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:52:23.114332 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:52:23.115919 (+ 1587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:52:23.116260 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:52:23.116261 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:52:23.116261 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:52:23.116312 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:52:23.116586 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:52:23.116586 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:52:23.116586 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:52:23.116686 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:52:23.116773 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 22:52:23.117024 (+ 251us) server_negotiation.cc:299] Negotiation successful
0806 22:52:23.117176 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 22:52:23.117326 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 12616
I20260806 22:52:23.117412 12754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:23.108570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:45957 (local address 127.28.218.195:52107)
0806 22:52:23.108735 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0806 22:52:23.108778 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 22:52:23.109297 (+ 519us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:52:23.109609 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:52:23.109613 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:52:23.110199 (+ 586us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:52:23.110407 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:52:23.110412 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:52:23.111188 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:52:23.111189 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:52:23.112006 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:52:23.112009 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:52:23.112418 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:52:23.112420 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:52:23.112543 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:52:23.113010 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:52:23.113018 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:52:23.114244 (+ 1226us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:52:23.115988 (+ 1744us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:52:23.115993 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:52:23.115995 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:52:23.116194 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:52:23.116398 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:52:23.116399 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:52:23.116399 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:52:23.116519 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:52:23.116743 (+ 224us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:52:23.116747 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:52:23.116905 (+ 158us) client_negotiation.cc:769] Sending connection context
0806 22:52:23.116991 (+ 86us) client_negotiation.cc:240] Negotiation successful
0806 22:52:23.117163 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":84,"threads_started":1}
I20260806 22:52:23.118050 12752 heartbeater.cc:347] Connected to a master server at 127.28.218.254:45957
I20260806 22:52:23.118166 12752 heartbeater.cc:464] Registering TS with master...
I20260806 22:52:23.118381 12752 heartbeater.cc:511] Master 127.28.218.254:45957 requested a full tablet report, sending...
I20260806 22:52:23.118789 12269 ts_manager.cc:194] Registered new tserver with Master: 4233903adc864154b6565c00f5654b70 (127.28.218.195:45531)
I20260806 22:52:23.119215 12269 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.218.195@KRBTEST.COM'} at 127.28.218.195:52107
I20260806 22:52:23.124043 12472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:23.117718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38318 (local address 127.28.218.254:45957)
0806 22:52:23.117772 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0806 22:52:23.117775 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:52:23.117906 (+ 131us) server_negotiation.cc:407] Connection header received
0806 22:52:23.117975 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:52:23.117976 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:52:23.117985 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:52:23.118052 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:52:23.118837 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:23.119512 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:23.120181 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:23.120375 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:23.120430 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:52:23.121493 (+ 1063us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:52:23.121494 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:52:23.121494 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:52:23.121500 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:52:23.122950 (+ 1450us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:52:23.123311 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:52:23.123312 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:52:23.123312 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:52:23.123360 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:52:23.123504 (+ 144us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:52:23.123504 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:52:23.123505 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:52:23.123586 (+ 81us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:52:23.123699 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0806 22:52:23.123930 (+ 231us) server_negotiation.cc:299] Negotiation successful
0806 22:52:23.123988 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 22:52:23.124805 29547 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:52:23.127410 12472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:23.125179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38328 (local address 127.28.218.254:45957)
0806 22:52:23.125218 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0806 22:52:23.125220 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:52:23.125296 (+ 76us) server_negotiation.cc:407] Connection header received
0806 22:52:23.125370 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:52:23.125371 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:52:23.125396 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:52:23.125455 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:52:23.125546 (+ 91us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:23.126001 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:23.126683 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:23.126897 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:23.126954 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:52:23.127065 (+ 111us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:52:23.127212 (+ 147us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:52:23.127275 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 22:52:23.127309 (+ 34us) server_negotiation.cc:299] Negotiation successful
0806 22:52:23.127322 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 22:52:23.127705 12269 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:38328:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:52:23.129374 12269 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:52:23.131491 12399 tablet_service.cc:1467] Processing CreateTablet for tablet b2984fd22aef4f5885be76bbae247bc9 (DEFAULT_TABLE table=test-table [id=358d25180dc4407f8d67a48b96251214]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:52:23.131700 12399 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b2984fd22aef4f5885be76bbae247bc9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:52:23.133234 12356 tablet_bootstrap.cc:492] T b2984fd22aef4f5885be76bbae247bc9 P 7a4f5f00a15646a3ae17db2711fff3e0: Bootstrap starting.
I20260806 22:52:23.133569 12356 tablet_bootstrap.cc:654] T b2984fd22aef4f5885be76bbae247bc9 P 7a4f5f00a15646a3ae17db2711fff3e0: Neither blocks nor log segments found. Creating new log.
I20260806 22:52:23.134521 12356 tablet_bootstrap.cc:492] T b2984fd22aef4f5885be76bbae247bc9 P 7a4f5f00a15646a3ae17db2711fff3e0: No bootstrap required, opened a new log
I20260806 22:52:23.134572 12356 ts_tablet_manager.cc:1397] T b2984fd22aef4f5885be76bbae247bc9 P 7a4f5f00a15646a3ae17db2711fff3e0: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260806 22:52:23.134742 12356 raft_consensus.cc:348] T b2984fd22aef4f5885be76bbae247bc9 P 7a4f5f00a15646a3ae17db2711fff3e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4233903adc864154b6565c00f5654b70" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 45531 } } peers { permanent_uuid: "860a5cffbfbc4199a28144ec07d1fcd3" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 42589 } } peers { permanent_uuid: "7a4f5f00a15646a3ae17db2711fff3e0" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 34809 } }
I20260806 22:52:23.134806 12356 raft_consensus.cc:374] T b2984fd22aef4f5885be76bbae247bc9 P 7a4f5f00a15646a3ae17db2711fff3e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:52:23.134851 12356 raft_consensus.cc:729] T b2984fd22aef4f5885be76bbae247bc9 P 7a4f5f00a15646a3ae17db2711fff3e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a4f5f00a15646a3ae17db2711fff3e0, State: Initialized, Role: FOLLOWER
I20260806 22:52:23.134912 12356 consensus_queue.cc:260] T b2984fd22aef4f5885be76bbae247bc9 P 7a4f5f00a15646a3ae17db2711fff3e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4233903adc864154b6565c00f5654b70" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 45531 } } peers { permanent_uuid: "860a5cffbfbc4199a28144ec07d1fcd3" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 42589 } } peers { permanent_uuid: "7a4f5f00a15646a3ae17db2711fff3e0" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 34809 } }
I20260806 22:52:23.135018 12356 ts_tablet_manager.cc:1428] T b2984fd22aef4f5885be76bbae247bc9 P 7a4f5f00a15646a3ae17db2711fff3e0: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 22:52:23.135285 12758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:23.131421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54916 (local address 127.28.218.195:45531)
0806 22:52:23.131591 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0806 22:52:23.131595 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:52:23.131606 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:52:23.131640 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:52:23.131643 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:52:23.131658 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:52:23.131745 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:52:23.131868 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:23.132738 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:23.133861 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:23.134859 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:23.134943 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:52:23.135000 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 22:52:23.135169 (+ 169us) server_negotiation.cc:299] Negotiation successful
0806 22:52:23.135209 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":82,"threads_started":1}
I20260806 22:52:23.135308 12476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:23.131277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.195:45531 (local address 127.0.0.1:54916)
0806 22:52:23.131313 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0806 22:52:23.131322 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:52:23.131355 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:52:23.131760 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:52:23.131761 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:52:23.131763 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:52:23.131801 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:52:23.131802 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:52:23.132837 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:52:23.132838 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:52:23.133745 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:52:23.133748 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:52:23.134977 (+ 1229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:52:23.134978 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:52:23.135058 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:52:23.135079 (+ 21us) client_negotiation.cc:769] Sending connection context
0806 22:52:23.135153 (+ 74us) client_negotiation.cc:240] Negotiation successful
0806 22:52:23.135242 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21,"mutex_wait_us":39}
I20260806 22:52:23.135932 12681 tablet_service.cc:1467] Processing CreateTablet for tablet b2984fd22aef4f5885be76bbae247bc9 (DEFAULT_TABLE table=test-table [id=358d25180dc4407f8d67a48b96251214]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:52:23.136263 12681 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b2984fd22aef4f5885be76bbae247bc9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:52:23.137447 12757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:23.131387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.194:42589 (local address 127.0.0.1:59126)
0806 22:52:23.131476 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0806 22:52:23.131484 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 22:52:23.131549 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:52:23.134042 (+ 2493us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:52:23.134044 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:52:23.134045 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:52:23.134095 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:52:23.134097 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:52:23.135066 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:52:23.135067 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:52:23.136312 (+ 1245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:52:23.136315 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:52:23.137212 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:52:23.137213 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:52:23.137279 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:52:23.137283 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:52:23.137338 (+ 55us) client_negotiation.cc:240] Negotiation successful
0806 22:52:23.137378 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":78,"thread_start_us":44,"threads_started":1}
I20260806 22:52:23.137476 12759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:23.133255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59126 (local address 127.28.218.194:42589)
0806 22:52:23.133882 (+ 627us) server_negotiation.cc:206] Beginning negotiation
0806 22:52:23.133885 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:52:23.133892 (+ 7us) server_negotiation.cc:407] Connection header received
0806 22:52:23.133924 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:52:23.133925 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:52:23.133937 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:52:23.134044 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:52:23.134194 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:23.134975 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:23.136394 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:23.137106 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:23.137211 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:52:23.137264 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 22:52:23.137351 (+ 87us) server_negotiation.cc:299] Negotiation successful
0806 22:52:23.137394 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":598,"thread_start_us":86,"threads_started":1}
I20260806 22:52:23.138054 12542 tablet_service.cc:1467] Processing CreateTablet for tablet b2984fd22aef4f5885be76bbae247bc9 (DEFAULT_TABLE table=test-table [id=358d25180dc4407f8d67a48b96251214]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:52:23.138348 12542 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b2984fd22aef4f5885be76bbae247bc9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:52:23.139106 12761 tablet_bootstrap.cc:492] T b2984fd22aef4f5885be76bbae247bc9 P 4233903adc864154b6565c00f5654b70: Bootstrap starting.
I20260806 22:52:23.139685 12761 tablet_bootstrap.cc:654] T b2984fd22aef4f5885be76bbae247bc9 P 4233903adc864154b6565c00f5654b70: Neither blocks nor log segments found. Creating new log.
I20260806 22:52:23.140143 12761 log.cc:824] T b2984fd22aef4f5885be76bbae247bc9 P 4233903adc864154b6565c00f5654b70: Log is configured to *not* fsync() on all Append() calls
I20260806 22:52:23.140826 12761 tablet_bootstrap.cc:492] T b2984fd22aef4f5885be76bbae247bc9 P 4233903adc864154b6565c00f5654b70: No bootstrap required, opened a new log
I20260806 22:52:23.140903 12761 ts_tablet_manager.cc:1397] T b2984fd22aef4f5885be76bbae247bc9 P 4233903adc864154b6565c00f5654b70: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260806 22:52:23.141319 12763 tablet_bootstrap.cc:492] T b2984fd22aef4f5885be76bbae247bc9 P 860a5cffbfbc4199a28144ec07d1fcd3: Bootstrap starting.
I20260806 22:52:23.141952 12763 tablet_bootstrap.cc:654] T b2984fd22aef4f5885be76bbae247bc9 P 860a5cffbfbc4199a28144ec07d1fcd3: Neither blocks nor log segments found. Creating new log.
I20260806 22:52:23.142391 12763 log.cc:824] T b2984fd22aef4f5885be76bbae247bc9 P 860a5cffbfbc4199a28144ec07d1fcd3: Log is configured to *not* fsync() on all Append() calls
I20260806 22:52:23.142628 12761 raft_consensus.cc:348] T b2984fd22aef4f5885be76bbae247bc9 P 4233903adc864154b6565c00f5654b70 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4233903adc864154b6565c00f5654b70" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 45531 } } peers { permanent_uuid: "860a5cffbfbc4199a28144ec07d1fcd3" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 42589 } } peers { permanent_uuid: "7a4f5f00a15646a3ae17db2711fff3e0" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 34809 } }
I20260806 22:52:23.142796 12761 raft_consensus.cc:374] T b2984fd22aef4f5885be76bbae247bc9 P 4233903adc864154b6565c00f5654b70 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:52:23.142838 12761 raft_consensus.cc:729] T b2984fd22aef4f5885be76bbae247bc9 P 4233903adc864154b6565c00f5654b70 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4233903adc864154b6565c00f5654b70, State: Initialized, Role: FOLLOWER
I20260806 22:52:23.142980 12761 consensus_queue.cc:260] T b2984fd22aef4f5885be76bbae247bc9 P 4233903adc864154b6565c00f5654b70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4233903adc864154b6565c00f5654b70" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 45531 } } peers { permanent_uuid: "860a5cffbfbc4199a28144ec07d1fcd3" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 42589 } } peers { permanent_uuid: "7a4f5f00a15646a3ae17db2711fff3e0" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 34809 } }
I20260806 22:52:23.143255 12763 tablet_bootstrap.cc:492] T b2984fd22aef4f5885be76bbae247bc9 P 860a5cffbfbc4199a28144ec07d1fcd3: No bootstrap required, opened a new log
I20260806 22:52:23.143272 12761 ts_tablet_manager.cc:1428] T b2984fd22aef4f5885be76bbae247bc9 P 4233903adc864154b6565c00f5654b70: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260806 22:52:23.143287 12752 heartbeater.cc:503] Master 127.28.218.254:45957 was elected leader, sending a full tablet report...
I20260806 22:52:23.143345 12763 ts_tablet_manager.cc:1397] T b2984fd22aef4f5885be76bbae247bc9 P 860a5cffbfbc4199a28144ec07d1fcd3: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:52:23.145211 12763 raft_consensus.cc:348] T b2984fd22aef4f5885be76bbae247bc9 P 860a5cffbfbc4199a28144ec07d1fcd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4233903adc864154b6565c00f5654b70" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 45531 } } peers { permanent_uuid: "860a5cffbfbc4199a28144ec07d1fcd3" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 42589 } } peers { permanent_uuid: "7a4f5f00a15646a3ae17db2711fff3e0" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 34809 } }
I20260806 22:52:23.145358 12763 raft_consensus.cc:374] T b2984fd22aef4f5885be76bbae247bc9 P 860a5cffbfbc4199a28144ec07d1fcd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:52:23.145382 12763 raft_consensus.cc:729] T b2984fd22aef4f5885be76bbae247bc9 P 860a5cffbfbc4199a28144ec07d1fcd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 860a5cffbfbc4199a28144ec07d1fcd3, State: Initialized, Role: FOLLOWER
I20260806 22:52:23.145521 12763 consensus_queue.cc:260] T b2984fd22aef4f5885be76bbae247bc9 P 860a5cffbfbc4199a28144ec07d1fcd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4233903adc864154b6565c00f5654b70" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 45531 } } peers { permanent_uuid: "860a5cffbfbc4199a28144ec07d1fcd3" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 42589 } } peers { permanent_uuid: "7a4f5f00a15646a3ae17db2711fff3e0" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 34809 } }
I20260806 22:52:23.145798 12763 ts_tablet_manager.cc:1428] T b2984fd22aef4f5885be76bbae247bc9 P 860a5cffbfbc4199a28144ec07d1fcd3: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 22:52:23.145814 12613 heartbeater.cc:503] Master 127.28.218.254:45957 was elected leader, sending a full tablet report...
I20260806 22:52:23.210253 12474 raft_consensus.cc:482] T b2984fd22aef4f5885be76bbae247bc9 P 7a4f5f00a15646a3ae17db2711fff3e0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:52:23.210384 12474 raft_consensus.cc:504] T b2984fd22aef4f5885be76bbae247bc9 P 7a4f5f00a15646a3ae17db2711fff3e0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4233903adc864154b6565c00f5654b70" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 45531 } } peers { permanent_uuid: "860a5cffbfbc4199a28144ec07d1fcd3" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 42589 } } peers { permanent_uuid: "7a4f5f00a15646a3ae17db2711fff3e0" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 34809 } }
I20260806 22:52:23.210791 12474 leader_election.cc:288] T b2984fd22aef4f5885be76bbae247bc9 P 7a4f5f00a15646a3ae17db2711fff3e0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4233903adc864154b6565c00f5654b70 (127.28.218.195:45531), 860a5cffbfbc4199a28144ec07d1fcd3 (127.28.218.194:42589)
I20260806 22:52:23.213816 12758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:23.211060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.193:51963 (local address 127.28.218.195:45531)
0806 22:52:23.211140 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0806 22:52:23.211142 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:52:23.211153 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:52:23.211240 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:52:23.211242 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:52:23.211255 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:52:23.211303 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:52:23.211563 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:23.212112 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:23.213030 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:23.213432 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:23.213487 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:52:23.213494 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0806 22:52:23.213701 (+ 207us) server_negotiation.cc:299] Negotiation successful
0806 22:52:23.213745 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 22:52:23.213816 12471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:23.210980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.195:45531 (local address 127.28.218.193:51963)
0806 22:52:23.211041 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0806 22:52:23.211053 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:52:23.211106 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:52:23.211359 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:52:23.211360 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:52:23.211362 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:52:23.211466 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:52:23.211469 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:52:23.212195 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:52:23.212196 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:52:23.212936 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:52:23.212938 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:52:23.213515 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:52:23.213516 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:52:23.213605 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:52:23.213635 (+ 30us) client_negotiation.cc:769] Sending connection context
0806 22:52:23.213679 (+ 44us) client_negotiation.cc:240] Negotiation successful
0806 22:52:23.213734 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260806 22:52:23.213855 12759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:23.211287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.193:44999 (local address 127.28.218.194:42589)
0806 22:52:23.211347 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0806 22:52:23.211349 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:52:23.211431 (+ 82us) server_negotiation.cc:407] Connection header received
0806 22:52:23.211481 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:52:23.211482 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:52:23.211492 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:52:23.211547 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:52:23.211716 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:23.212186 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:23.213103 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:23.213500 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:23.213572 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:52:23.213581 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 22:52:23.213749 (+ 168us) server_negotiation.cc:299] Negotiation successful
0806 22:52:23.213788 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260806 22:52:23.213915 12768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:23.211110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.194:42589 (local address 127.28.218.193:44999)
0806 22:52:23.211342 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0806 22:52:23.211352 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:52:23.211418 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:52:23.211565 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:52:23.211566 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:52:23.211567 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:52:23.211642 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:52:23.211644 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:52:23.212263 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:52:23.212264 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:52:23.213005 (+ 741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:52:23.213007 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:52:23.213597 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:52:23.213598 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:52:23.213652 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:52:23.213655 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:52:23.213726 (+ 71us) client_negotiation.cc:240] Negotiation successful
0806 22:52:23.213760 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":164,"threads_started":1}
I20260806 22:52:23.214306 12702 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2984fd22aef4f5885be76bbae247bc9" candidate_uuid: "7a4f5f00a15646a3ae17db2711fff3e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4233903adc864154b6565c00f5654b70" is_pre_election: true
I20260806 22:52:23.214304 12564 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2984fd22aef4f5885be76bbae247bc9" candidate_uuid: "7a4f5f00a15646a3ae17db2711fff3e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "860a5cffbfbc4199a28144ec07d1fcd3" is_pre_election: true
I20260806 22:52:23.214464 12564 raft_consensus.cc:2454] T b2984fd22aef4f5885be76bbae247bc9 P 860a5cffbfbc4199a28144ec07d1fcd3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a4f5f00a15646a3ae17db2711fff3e0 in term 0.
I20260806 22:52:23.214464 12702 raft_consensus.cc:2454] T b2984fd22aef4f5885be76bbae247bc9 P 4233903adc864154b6565c00f5654b70 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a4f5f00a15646a3ae17db2711fff3e0 in term 0.
I20260806 22:52:23.214670 12350 leader_election.cc:302] T b2984fd22aef4f5885be76bbae247bc9 P 7a4f5f00a15646a3ae17db2711fff3e0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4233903adc864154b6565c00f5654b70, 7a4f5f00a15646a3ae17db2711fff3e0; no voters:
I20260806 22:52:23.214818 12474 raft_consensus.cc:2781] T b2984fd22aef4f5885be76bbae247bc9 P 7a4f5f00a15646a3ae17db2711fff3e0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:52:23.214890 12474 raft_consensus.cc:482] T b2984fd22aef4f5885be76bbae247bc9 P 7a4f5f00a15646a3ae17db2711fff3e0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:52:23.214907 12474 raft_consensus.cc:3037] T b2984fd22aef4f5885be76bbae247bc9 P 7a4f5f00a15646a3ae17db2711fff3e0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:52:23.215754 12474 raft_consensus.cc:504] T b2984fd22aef4f5885be76bbae247bc9 P 7a4f5f00a15646a3ae17db2711fff3e0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4233903adc864154b6565c00f5654b70" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 45531 } } peers { permanent_uuid: "860a5cffbfbc4199a28144ec07d1fcd3" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 42589 } } peers { permanent_uuid: "7a4f5f00a15646a3ae17db2711fff3e0" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 34809 } }
I20260806 22:52:23.215881 12474 leader_election.cc:288] T b2984fd22aef4f5885be76bbae247bc9 P 7a4f5f00a15646a3ae17db2711fff3e0 [CANDIDATE]: Term 1 election: Requested vote from peers 4233903adc864154b6565c00f5654b70 (127.28.218.195:45531), 860a5cffbfbc4199a28144ec07d1fcd3 (127.28.218.194:42589)
I20260806 22:52:23.216084 12702 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2984fd22aef4f5885be76bbae247bc9" candidate_uuid: "7a4f5f00a15646a3ae17db2711fff3e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4233903adc864154b6565c00f5654b70"
I20260806 22:52:23.216080 12564 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2984fd22aef4f5885be76bbae247bc9" candidate_uuid: "7a4f5f00a15646a3ae17db2711fff3e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "860a5cffbfbc4199a28144ec07d1fcd3"
I20260806 22:52:23.216147 12702 raft_consensus.cc:3037] T b2984fd22aef4f5885be76bbae247bc9 P 4233903adc864154b6565c00f5654b70 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:52:23.216182 12564 raft_consensus.cc:3037] T b2984fd22aef4f5885be76bbae247bc9 P 860a5cffbfbc4199a28144ec07d1fcd3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:52:23.216856 12702 raft_consensus.cc:2454] T b2984fd22aef4f5885be76bbae247bc9 P 4233903adc864154b6565c00f5654b70 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a4f5f00a15646a3ae17db2711fff3e0 in term 1.
I20260806 22:52:23.216970 12564 raft_consensus.cc:2454] T b2984fd22aef4f5885be76bbae247bc9 P 860a5cffbfbc4199a28144ec07d1fcd3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a4f5f00a15646a3ae17db2711fff3e0 in term 1.
I20260806 22:52:23.217037 12350 leader_election.cc:302] T b2984fd22aef4f5885be76bbae247bc9 P 7a4f5f00a15646a3ae17db2711fff3e0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4233903adc864154b6565c00f5654b70, 7a4f5f00a15646a3ae17db2711fff3e0; no voters:
I20260806 22:52:23.217118 12474 raft_consensus.cc:2781] T b2984fd22aef4f5885be76bbae247bc9 P 7a4f5f00a15646a3ae17db2711fff3e0 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:52:23.217248 12474 raft_consensus.cc:686] T b2984fd22aef4f5885be76bbae247bc9 P 7a4f5f00a15646a3ae17db2711fff3e0 [term 1 LEADER]: Becoming Leader. State: Replica: 7a4f5f00a15646a3ae17db2711fff3e0, State: Running, Role: LEADER
I20260806 22:52:23.217300 12474 consensus_queue.cc:237] T b2984fd22aef4f5885be76bbae247bc9 P 7a4f5f00a15646a3ae17db2711fff3e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4233903adc864154b6565c00f5654b70" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 45531 } } peers { permanent_uuid: "860a5cffbfbc4199a28144ec07d1fcd3" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 42589 } } peers { permanent_uuid: "7a4f5f00a15646a3ae17db2711fff3e0" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 34809 } }
I20260806 22:52:23.218060 12269 catalog_manager.cc:5495] T b2984fd22aef4f5885be76bbae247bc9 P 7a4f5f00a15646a3ae17db2711fff3e0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7a4f5f00a15646a3ae17db2711fff3e0 (127.28.218.193). New cstate: current_term: 1 leader_uuid: "7a4f5f00a15646a3ae17db2711fff3e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4233903adc864154b6565c00f5654b70" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 45531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "860a5cffbfbc4199a28144ec07d1fcd3" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 42589 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a4f5f00a15646a3ae17db2711fff3e0" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 34809 } health_report { overall_health: HEALTHY } } }
W20260806 22:52:23.224958 12614 tablet.cc:2369] T b2984fd22aef4f5885be76bbae247bc9 P 860a5cffbfbc4199a28144ec07d1fcd3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:52:23.228353 12477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:23.226223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47150 (local address 127.28.218.193:34809)
0806 22:52:23.226282 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0806 22:52:23.226284 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:52:23.226317 (+ 33us) server_negotiation.cc:407] Connection header received
0806 22:52:23.226388 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:52:23.226389 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:52:23.226402 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:52:23.226457 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:52:23.226575 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:23.227010 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:23.227744 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:23.227930 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:23.227984 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:52:23.228095 (+ 111us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:52:23.228175 (+ 80us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:52:23.228222 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0806 22:52:23.228259 (+ 37us) server_negotiation.cc:299] Negotiation successful
0806 22:52:23.228274 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 22:52:23.229578 12564 raft_consensus.cc:1261] T b2984fd22aef4f5885be76bbae247bc9 P 860a5cffbfbc4199a28144ec07d1fcd3 [term 1 FOLLOWER]: Refusing update from remote peer 7a4f5f00a15646a3ae17db2711fff3e0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:52:23.229648 12702 raft_consensus.cc:1261] T b2984fd22aef4f5885be76bbae247bc9 P 4233903adc864154b6565c00f5654b70 [term 1 FOLLOWER]: Refusing update from remote peer 7a4f5f00a15646a3ae17db2711fff3e0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:52:23.229890 12474 consensus_queue.cc:1035] T b2984fd22aef4f5885be76bbae247bc9 P 7a4f5f00a15646a3ae17db2711fff3e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "860a5cffbfbc4199a28144ec07d1fcd3" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 42589 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:52:23.229998 12769 consensus_queue.cc:1035] T b2984fd22aef4f5885be76bbae247bc9 P 7a4f5f00a15646a3ae17db2711fff3e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4233903adc864154b6565c00f5654b70" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 45531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:52:23.232697 12773 mvcc.cc:205] Tried to move back new op lower bound from 7315688420266434560 to 7315688420218675200. Current Snapshot: MvccSnapshot[applied={T|T < 7315688420266434560}]
I20260806 22:52:23.234603 12268 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:38328:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:52:23.234699 12268 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:38328:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:52:23.235411 12268 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d312610ebc6a4ab3aaec82261c28aa09: Sending DeleteTablet for 3 replicas of tablet b2984fd22aef4f5885be76bbae247bc9
I20260806 22:52:23.235677 12399 tablet_service.cc:1514] Processing DeleteTablet for tablet b2984fd22aef4f5885be76bbae247bc9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:52:23 UTC) from {username='kudu', principal='kudu/127.28.218.254@KRBTEST.COM'} at 127.0.0.1:47142
I20260806 22:52:23.235772 12478 tablet_replica.cc:330] T b2984fd22aef4f5885be76bbae247bc9 P 7a4f5f00a15646a3ae17db2711fff3e0: stopping tablet replica
I20260806 22:52:23.235767 12542 tablet_service.cc:1514] Processing DeleteTablet for tablet b2984fd22aef4f5885be76bbae247bc9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:52:23 UTC) from {username='kudu', principal='kudu/127.28.218.254@KRBTEST.COM'} at 127.0.0.1:59126
I20260806 22:52:23.235826 12478 raft_consensus.cc:2229] T b2984fd22aef4f5885be76bbae247bc9 P 7a4f5f00a15646a3ae17db2711fff3e0 [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:52:23.235903 12478 raft_consensus.cc:2258] T b2984fd22aef4f5885be76bbae247bc9 P 7a4f5f00a15646a3ae17db2711fff3e0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:52:23.235993 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 12313
I20260806 22:52:23.236028 12782 tablet_replica.cc:330] T b2984fd22aef4f5885be76bbae247bc9 P 860a5cffbfbc4199a28144ec07d1fcd3: stopping tablet replica
I20260806 22:52:23.236193 12782 raft_consensus.cc:2229] T b2984fd22aef4f5885be76bbae247bc9 P 860a5cffbfbc4199a28144ec07d1fcd3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:52:23.236325 12782 raft_consensus.cc:2258] T b2984fd22aef4f5885be76bbae247bc9 P 860a5cffbfbc4199a28144ec07d1fcd3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:52:23.237135 12782 ts_tablet_manager.cc:1905] T b2984fd22aef4f5885be76bbae247bc9 P 860a5cffbfbc4199a28144ec07d1fcd3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:52:23.238597 12782 ts_tablet_manager.cc:1918] T b2984fd22aef4f5885be76bbae247bc9 P 860a5cffbfbc4199a28144ec07d1fcd3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:52:23.238683 12782 log.cc:1196] T b2984fd22aef4f5885be76bbae247bc9 P 860a5cffbfbc4199a28144ec07d1fcd3: Deleting WAL directory at /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-1/wal/wals/b2984fd22aef4f5885be76bbae247bc9
I20260806 22:52:23.238947 12782 ts_tablet_manager.cc:1939] T b2984fd22aef4f5885be76bbae247bc9 P 860a5cffbfbc4199a28144ec07d1fcd3: Deleting consensus metadata
I20260806 22:52:23.239777 12252 catalog_manager.cc:4841] TS 860a5cffbfbc4199a28144ec07d1fcd3 (127.28.218.194:42589): tablet b2984fd22aef4f5885be76bbae247bc9 (table test-table [id=358d25180dc4407f8d67a48b96251214]) successfully deleted
I20260806 22:52:23.243280 12681 tablet_service.cc:1514] Processing DeleteTablet for tablet b2984fd22aef4f5885be76bbae247bc9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:52:23 UTC) from {username='kudu', principal='kudu/127.28.218.254@KRBTEST.COM'} at 127.0.0.1:54916
I20260806 22:52:23.243520 12783 tablet_replica.cc:330] T b2984fd22aef4f5885be76bbae247bc9 P 4233903adc864154b6565c00f5654b70: stopping tablet replica
W20260806 22:52:23.243495 12255 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.28.218.193:34809 (error 108)
I20260806 22:52:23.243611 12783 raft_consensus.cc:2229] T b2984fd22aef4f5885be76bbae247bc9 P 4233903adc864154b6565c00f5654b70 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:52:23.243708 12783 raft_consensus.cc:2258] T b2984fd22aef4f5885be76bbae247bc9 P 4233903adc864154b6565c00f5654b70 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260806 22:52:23.244009 12757 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:52:23.243858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.193:34809 (local address 127.0.0.1:47160)
0806 22:52:23.243913 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0806 22:52:23.243924 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:52:23.243961 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.193:34809: BlockingWrite error: write error: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":35}
I20260806 22:52:23.244266 12783 ts_tablet_manager.cc:1905] T b2984fd22aef4f5885be76bbae247bc9 P 4233903adc864154b6565c00f5654b70: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:52:23.244288 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 12475
W20260806 22:52:23.244133 12255 catalog_manager.cc:4584] TS 7a4f5f00a15646a3ae17db2711fff3e0 (127.28.218.193:34809): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b2984fd22aef4f5885be76bbae247bc9: Network error: Client connection negotiation failed: client connection to 127.28.218.193:34809: BlockingWrite error: write error: Connection reset by peer (error 104)
I20260806 22:52:23.246321 12783 ts_tablet_manager.cc:1918] T b2984fd22aef4f5885be76bbae247bc9 P 4233903adc864154b6565c00f5654b70: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:52:23.246416 12783 log.cc:1196] T b2984fd22aef4f5885be76bbae247bc9 P 4233903adc864154b6565c00f5654b70: Deleting WAL directory at /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-2/wal/wals/b2984fd22aef4f5885be76bbae247bc9
I20260806 22:52:23.246794 12783 ts_tablet_manager.cc:1939] T b2984fd22aef4f5885be76bbae247bc9 P 4233903adc864154b6565c00f5654b70: Deleting consensus metadata
I20260806 22:52:23.248185 12252 catalog_manager.cc:4841] TS 4233903adc864154b6565c00f5654b70 (127.28.218.195:45531): tablet b2984fd22aef4f5885be76bbae247bc9 (table test-table [id=358d25180dc4407f8d67a48b96251214]) successfully deleted
I20260806 22:52:23.251919 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 12616
I20260806 22:52:23.257493 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 12234
I20260806 22:52:23.263718 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.218.254:45957
--webserver_interface=127.28.218.254
--webserver_port=33089
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab
--principal=kudu/127.28.218.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.218.212:41905
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.218.254:45957
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:52:23.346681 12785 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:52:23.346901 12785 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:52:23.346925 12785 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:52:23.348033 12785 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:52:23.348069 12785 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:52:23.348083 12785 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:52:23.348096 12785 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:52:23.348109 12785 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:52:23.349500 12785 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:41905
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.218.254:45957
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab
--principal=kudu/127.28.218.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.218.254:45957
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.218.254
--webserver_port=33089
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12785
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:52:23.349692 12785 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:52:23.349895 12785 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:52:23.354650 12791 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:52:23.354657 12790 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:52:23.354842 12785 server_base.cc:1034] running on GCE node
W20260806 22:52:23.354698 12793 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:52:23.355106 12785 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:52:23.355453 12785 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:52:23.356613 12785 hybrid_clock.cc:648] HybridClock initialized: now 1786056743356611 us; error 34 us; skew 500 ppm
Aug 06 22:52:23 dist-test-slave-1xrh krb5kdc[11659](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056743, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:52:23.359062 12785 init.cc:377] Logged in from keytab as kudu/127.28.218.254@KRBTEST.COM (short username kudu)
I20260806 22:52:23.359556 12785 webserver.cc:460] Webserver started at http://127.28.218.254:33089/ using document root <none> and password file <none>
I20260806 22:52:23.359774 12785 fs_manager.cc:359] Metadata directory not provided
I20260806 22:52:23.359843 12785 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:52:23.361166 12785 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:52:23.361860 12800 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:52:23.362035 12785 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:52:23.362133 12785 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/master-0/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/master-0/wal
uuid: "d312610ebc6a4ab3aaec82261c28aa09"
format_stamp: "Formatted at 2026-08-06 22:52:21 on dist-test-slave-1xrh"
server_key: "299f120946a4497f663707ffd8d20458"
server_key_iv: "375f6ffbd4a558e198668ceaa76da574"
server_key_version: "encryptionkey@0"
I20260806 22:52:23.362497 12785 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:52:23.388408 12785 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:52:23.390717 12785 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:52:23.390894 12785 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:52:23.395040 12785 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.254:45957
I20260806 22:52:23.395073 12856 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.254:45957 every 8 connection(s)
I20260806 22:52:23.395484 12785 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
I20260806 22:52:23.397032 12857 sys_catalog.cc:263] Verifying existing consensus state
I20260806 22:52:23.397615 12857 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d312610ebc6a4ab3aaec82261c28aa09: Bootstrap starting.
I20260806 22:52:23.398708 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 12785
I20260806 22:52:23.398976 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.218.193:34809
--local_ip_for_outbound_sockets=127.28.218.193
--tserver_master_addrs=127.28.218.254:45957
--webserver_port=44957
--webserver_interface=127.28.218.193
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.193.keytab
--principal=kudu/127.28.218.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.218.212:41905
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:52:23.408012 12857 log.cc:824] T 00000000000000000000000000000000 P d312610ebc6a4ab3aaec82261c28aa09: Log is configured to *not* fsync() on all Append() calls
I20260806 22:52:23.411605 12857 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d312610ebc6a4ab3aaec82261c28aa09: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 22:52:23.411954 12857 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d312610ebc6a4ab3aaec82261c28aa09: Bootstrap complete.
I20260806 22:52:23.415424 12857 raft_consensus.cc:348] T 00000000000000000000000000000000 P d312610ebc6a4ab3aaec82261c28aa09 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d312610ebc6a4ab3aaec82261c28aa09" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 45957 } }
I20260806 22:52:23.415741 12857 raft_consensus.cc:729] T 00000000000000000000000000000000 P d312610ebc6a4ab3aaec82261c28aa09 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: d312610ebc6a4ab3aaec82261c28aa09, State: Initialized, Role: FOLLOWER
I20260806 22:52:23.415902 12857 consensus_queue.cc:260] T 00000000000000000000000000000000 P d312610ebc6a4ab3aaec82261c28aa09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d312610ebc6a4ab3aaec82261c28aa09" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 45957 } }
I20260806 22:52:23.415964 12857 raft_consensus.cc:388] T 00000000000000000000000000000000 P d312610ebc6a4ab3aaec82261c28aa09 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:52:23.416003 12857 raft_consensus.cc:482] T 00000000000000000000000000000000 P d312610ebc6a4ab3aaec82261c28aa09 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:52:23.416044 12857 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d312610ebc6a4ab3aaec82261c28aa09 [term 2 FOLLOWER]: Advancing to term 3
I20260806 22:52:23.417383 12857 raft_consensus.cc:504] T 00000000000000000000000000000000 P d312610ebc6a4ab3aaec82261c28aa09 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d312610ebc6a4ab3aaec82261c28aa09" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 45957 } }
I20260806 22:52:23.417501 12857 leader_election.cc:302] T 00000000000000000000000000000000 P d312610ebc6a4ab3aaec82261c28aa09 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d312610ebc6a4ab3aaec82261c28aa09; no voters:
I20260806 22:52:23.417656 12857 leader_election.cc:288] T 00000000000000000000000000000000 P d312610ebc6a4ab3aaec82261c28aa09 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260806 22:52:23.417789 12861 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d312610ebc6a4ab3aaec82261c28aa09 [term 3 FOLLOWER]: Leader election won for term 3
I20260806 22:52:23.418067 12861 raft_consensus.cc:686] T 00000000000000000000000000000000 P d312610ebc6a4ab3aaec82261c28aa09 [term 3 LEADER]: Becoming Leader. State: Replica: d312610ebc6a4ab3aaec82261c28aa09, State: Running, Role: LEADER
I20260806 22:52:23.418248 12861 consensus_queue.cc:237] T 00000000000000000000000000000000 P d312610ebc6a4ab3aaec82261c28aa09 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d312610ebc6a4ab3aaec82261c28aa09" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 45957 } }
I20260806 22:52:23.417903 12857 sys_catalog.cc:564] T 00000000000000000000000000000000 P d312610ebc6a4ab3aaec82261c28aa09 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:52:23.418566 12864 sys_catalog.cc:455] T 00000000000000000000000000000000 P d312610ebc6a4ab3aaec82261c28aa09 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d312610ebc6a4ab3aaec82261c28aa09. Latest consensus state: current_term: 3 leader_uuid: "d312610ebc6a4ab3aaec82261c28aa09" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d312610ebc6a4ab3aaec82261c28aa09" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 45957 } } }
I20260806 22:52:23.418572 12863 sys_catalog.cc:455] T 00000000000000000000000000000000 P d312610ebc6a4ab3aaec82261c28aa09 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "d312610ebc6a4ab3aaec82261c28aa09" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d312610ebc6a4ab3aaec82261c28aa09" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 45957 } } }
I20260806 22:52:23.418653 12864 sys_catalog.cc:458] T 00000000000000000000000000000000 P d312610ebc6a4ab3aaec82261c28aa09 [sys.catalog]: This master's current role is: LEADER
I20260806 22:52:23.418697 12863 sys_catalog.cc:458] T 00000000000000000000000000000000 P d312610ebc6a4ab3aaec82261c28aa09 [sys.catalog]: This master's current role is: LEADER
I20260806 22:52:23.418838 12866 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:52:23.419615 12866 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:52:23.419737 12866 catalog_manager.cc:1259] Loaded cluster ID: 8c5f4590e8804d2dbfed5c3f208eb9e0
I20260806 22:52:23.419777 12866 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260806 22:52:23.420102 12866 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f1e695319ad google::LogMessage::Fail() at ??:0
@ 0x7f1e6953488a google::LogMessage::SendToLog() at ??:0
@ 0x7f1e695314de google::LogMessage::Flush() at ??:0
@ 0x7f1e69532a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f1e714520f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f1e7145b8b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f1e6b3ff0d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f1e6b3f4954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f1e6c2586db start_thread at ??:0
@ 0x7f1e6802f61f clone at ??:0
W20260806 22:52:23.497427 12859 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:52:23.497640 12859 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:52:23.497680 12859 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:52:23.498788 12859 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:52:23.498850 12859 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:52:23.498895 12859 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.218.193
I20260806 22:52:23.500347 12859 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:41905
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.193.keytab
--principal=kudu/127.28.218.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.218.193:34809
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.218.193
--webserver_port=44957
--tserver_master_addrs=127.28.218.254:45957
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12859
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.218.193
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:52:23.500633 12859 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:52:23.500857 12859 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:52:23.506692 12897 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:52:23.506706 12895 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:52:23.506882 12859 server_base.cc:1034] running on GCE node
W20260806 22:52:23.507040 12894 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:52:23.507292 12859 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:52:23.507578 12859 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:52:23.508764 12859 hybrid_clock.cc:648] HybridClock initialized: now 1786056743508749 us; error 28 us; skew 500 ppm
Aug 06 22:52:23 dist-test-slave-1xrh krb5kdc[11659](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056743, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:52:23.511720 12859 init.cc:377] Logged in from keytab as kudu/127.28.218.193@KRBTEST.COM (short username kudu)
I20260806 22:52:23.512195 12859 webserver.cc:460] Webserver started at http://127.28.218.193:44957/ using document root <none> and password file <none>
I20260806 22:52:23.512441 12859 fs_manager.cc:359] Metadata directory not provided
I20260806 22:52:23.512491 12859 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:52:23.513749 12859 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:52:23.514424 12904 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:52:23.514572 12859 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:52:23.514678 12859 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-0/wal
uuid: "7a4f5f00a15646a3ae17db2711fff3e0"
format_stamp: "Formatted at 2026-08-06 22:52:21 on dist-test-slave-1xrh"
server_key: "649619ee3e06e37c564b970f1879a0a3"
server_key_iv: "d920b093ce9032370d9745a810cb0925"
server_key_version: "encryptionkey@0"
I20260806 22:52:23.514979 12859 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:52:23.534880 12859 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:52:23.537034 12859 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:52:23.537182 12859 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:52:23.537408 12859 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:52:23.537911 12911 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 22:52:23.539851 12859 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 22:52:23.539942 12859 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260806 22:52:23.539995 12859 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 22:52:23.540654 12859 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 22:52:23.540716 12859 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260806 22:52:23.540776 12911 tablet_bootstrap.cc:492] T b2984fd22aef4f5885be76bbae247bc9 P 7a4f5f00a15646a3ae17db2711fff3e0: Bootstrap starting.
I20260806 22:52:23.548897 12859 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.193:34809
I20260806 22:52:23.549019 13023 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.193:34809 every 8 connection(s)
I20260806 22:52:23.549333 12859 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-0/data/info.pb
I20260806 22:52:23.551810 12911 log.cc:824] T b2984fd22aef4f5885be76bbae247bc9 P 7a4f5f00a15646a3ae17db2711fff3e0: Log is configured to *not* fsync() on all Append() calls
I20260806 22:52:23.554452 12911 tablet_bootstrap.cc:492] T b2984fd22aef4f5885be76bbae247bc9 P 7a4f5f00a15646a3ae17db2711fff3e0: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 22:52:23.554746 12911 tablet_bootstrap.cc:492] T b2984fd22aef4f5885be76bbae247bc9 P 7a4f5f00a15646a3ae17db2711fff3e0: Bootstrap complete.
I20260806 22:52:23.555727 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 12859
I20260806 22:52:23.556007 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.218.194:42589
--local_ip_for_outbound_sockets=127.28.218.194
--tserver_master_addrs=127.28.218.254:45957
--webserver_port=36183
--webserver_interface=127.28.218.194
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.194.keytab
--principal=kudu/127.28.218.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.218.212:41905
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:52:23.556020 12911 ts_tablet_manager.cc:1397] T b2984fd22aef4f5885be76bbae247bc9 P 7a4f5f00a15646a3ae17db2711fff3e0: Time spent bootstrapping tablet: real 0.015s user 0.004s sys 0.009s
I20260806 22:52:23.557384 12911 raft_consensus.cc:348] T b2984fd22aef4f5885be76bbae247bc9 P 7a4f5f00a15646a3ae17db2711fff3e0 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4233903adc864154b6565c00f5654b70" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 45531 } } peers { permanent_uuid: "860a5cffbfbc4199a28144ec07d1fcd3" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 42589 } } peers { permanent_uuid: "7a4f5f00a15646a3ae17db2711fff3e0" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 34809 } }
I20260806 22:52:23.557690 12911 raft_consensus.cc:729] T b2984fd22aef4f5885be76bbae247bc9 P 7a4f5f00a15646a3ae17db2711fff3e0 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a4f5f00a15646a3ae17db2711fff3e0, State: Initialized, Role: FOLLOWER
I20260806 22:52:23.557821 12911 consensus_queue.cc:260] T b2984fd22aef4f5885be76bbae247bc9 P 7a4f5f00a15646a3ae17db2711fff3e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4233903adc864154b6565c00f5654b70" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 45531 } } peers { permanent_uuid: "860a5cffbfbc4199a28144ec07d1fcd3" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 42589 } } peers { permanent_uuid: "7a4f5f00a15646a3ae17db2711fff3e0" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 34809 } }
I20260806 22:52:23.558081 12911 ts_tablet_manager.cc:1428] T b2984fd22aef4f5885be76bbae247bc9 P 7a4f5f00a15646a3ae17db2711fff3e0: Time spent starting tablet: real 0.002s user 0.001s sys 0.002s
W20260806 22:52:23.606815 13026 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:52:23.550124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:45957 (local address 127.28.218.193:36059)
0806 22:52:23.550332 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0806 22:52:23.550370 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 22:52:23.550877 (+ 507us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:52:23.606601 (+ 55724us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:45957: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":102,"threads_started":1}
W20260806 22:52:23.607090 13024 heartbeater.cc:644] Failed to heartbeat to 127.28.218.254:45957 (0 consecutive failures): Network error: Failed to ping master at 127.28.218.254:45957: Client connection negotiation failed: client connection to 127.28.218.254:45957: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 22:52:23.607496 13026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:23.607363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:45957 (local address 127.28.218.193:42511)
0806 22:52:23.607417 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0806 22:52:23.607470 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:45957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260806 22:52:23.607849 13026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:23.607753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:45957 (local address 127.28.218.193:47253)
0806 22:52:23.607798 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0806 22:52:23.607825 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:45957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260806 22:52:23.607969 13024 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 22:52:23.638537 13028 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:52:23.638715 13028 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:52:23.638737 13028 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:52:23.639917 13028 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:52:23.639966 13028 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:52:23.640020 13028 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.218.194
I20260806 22:52:23.641431 13028 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:41905
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.194.keytab
--principal=kudu/127.28.218.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.218.194:42589
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.218.194
--webserver_port=36183
--tserver_master_addrs=127.28.218.254:45957
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13028
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.218.194
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:52:23.641659 13028 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:52:23.641877 13028 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:52:23.646685 13037 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:52:23.646659 13034 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:52:23.646665 13035 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:52:23.646927 13028 server_base.cc:1034] running on GCE node
I20260806 22:52:23.647092 13028 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:52:23.647329 13028 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:52:23.648463 13028 hybrid_clock.cc:648] HybridClock initialized: now 1786056743648453 us; error 28 us; skew 500 ppm
Aug 06 22:52:23 dist-test-slave-1xrh krb5kdc[11659](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056743, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:52:23.650913 13028 init.cc:377] Logged in from keytab as kudu/127.28.218.194@KRBTEST.COM (short username kudu)
I20260806 22:52:23.651389 13028 webserver.cc:460] Webserver started at http://127.28.218.194:36183/ using document root <none> and password file <none>
I20260806 22:52:23.651621 13028 fs_manager.cc:359] Metadata directory not provided
I20260806 22:52:23.651685 13028 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:52:23.652858 13028 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:52:23.653364 13044 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:52:23.653506 13028 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:52:23.653610 13028 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-1/wal
uuid: "860a5cffbfbc4199a28144ec07d1fcd3"
format_stamp: "Formatted at 2026-08-06 22:52:21 on dist-test-slave-1xrh"
server_key: "ea7cf74de7358c4022b4b16d2b569640"
server_key_iv: "c45dfeff539ec2dd568e793ed682e328"
server_key_version: "encryptionkey@0"
I20260806 22:52:23.653909 13028 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:52:23.700431 13028 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:52:23.702749 13028 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:52:23.702910 13028 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:52:23.703171 13028 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:52:23.703471 13028 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:52:23.703531 13028 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:52:23.703576 13028 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:52:23.703599 13028 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:52:23.709487 13028 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.194:42589
I20260806 22:52:23.709566 13161 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.194:42589 every 8 connection(s)
I20260806 22:52:23.709900 13028 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-1/data/info.pb
I20260806 22:52:23.710912 13164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:23.710554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:45957 (local address 127.28.218.194:40923)
0806 22:52:23.710710 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0806 22:52:23.710783 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:45957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":70,"threads_started":1}
W20260806 22:52:23.711336 13162 heartbeater.cc:644] Failed to heartbeat to 127.28.218.254:45957 (0 consecutive failures): Network error: Failed to ping master at 127.28.218.254:45957: Client connection negotiation failed: client connection to 127.28.218.254:45957: connect: Connection refused (error 111)
I20260806 22:52:23.711586 13164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:23.711505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:45957 (local address 127.28.218.194:50445)
0806 22:52:23.711545 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0806 22:52:23.711571 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:45957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260806 22:52:23.711887 13164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:23.711815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:45957 (local address 127.28.218.194:34523)
0806 22:52:23.711855 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0806 22:52:23.711875 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:45957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260806 22:52:23.711912 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 13028
W20260806 22:52:23.712010 13162 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 22:52:23.712153 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.218.195:45531
--local_ip_for_outbound_sockets=127.28.218.195
--tserver_master_addrs=127.28.218.254:45957
--webserver_port=41411
--webserver_interface=127.28.218.195
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.195.keytab
--principal=kudu/127.28.218.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.218.212:41905
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:52:23.792847 13165 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:52:23.793045 13165 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:52:23.793068 13165 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:52:23.794087 13165 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:52:23.794118 13165 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:52:23.794142 13165 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.218.195
I20260806 22:52:23.795477 13165 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:41905
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.195.keytab
--principal=kudu/127.28.218.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.218.195:45531
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.218.195
--webserver_port=41411
--tserver_master_addrs=127.28.218.254:45957
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13165
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.218.195
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:52:23.795644 13165 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:52:23.795805 13165 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:52:23.800681 13173 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:52:23.800674 13171 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:52:23.800747 13165 server_base.cc:1034] running on GCE node
W20260806 22:52:23.800859 13170 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:52:23.801087 13165 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:52:23.801321 13165 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:52:23.802481 13165 hybrid_clock.cc:648] HybridClock initialized: now 1786056743802468 us; error 25 us; skew 500 ppm
Aug 06 22:52:23 dist-test-slave-1xrh krb5kdc[11659](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056743, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:52:23.804917 13165 init.cc:377] Logged in from keytab as kudu/127.28.218.195@KRBTEST.COM (short username kudu)
I20260806 22:52:23.805313 13165 webserver.cc:460] Webserver started at http://127.28.218.195:41411/ using document root <none> and password file <none>
I20260806 22:52:23.805545 13165 fs_manager.cc:359] Metadata directory not provided
I20260806 22:52:23.805614 13165 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:52:23.806886 13165 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:52:23.807472 13180 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:52:23.807639 13165 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 22:52:23.807734 13165 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-2/wal
uuid: "4233903adc864154b6565c00f5654b70"
format_stamp: "Formatted at 2026-08-06 22:52:22 on dist-test-slave-1xrh"
server_key: "cf2c4923bd749fe655cf5a8c0ad4a6c6"
server_key_iv: "e4ba17445af6dc4914b6655b98d54357"
server_key_version: "encryptionkey@0"
I20260806 22:52:23.808043 13165 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:52:23.826998 13165 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:52:23.828729 13165 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:52:23.828884 13165 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:52:23.829110 13165 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:52:23.829406 13165 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:52:23.829468 13165 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:52:23.829509 13165 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:52:23.829540 13165 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:52:23.835306 13165 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.195:45531
I20260806 22:52:23.835343 13300 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.195:45531 every 8 connection(s)
I20260806 22:52:23.835680 13165 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056586788339-29547-0/minicluster-data/ts-2/data/info.pb
I20260806 22:52:23.836699 13303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:23.836336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:45957 (local address 127.28.218.195:57361)
0806 22:52:23.836477 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0806 22:52:23.836559 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:45957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":65,"threads_started":1}
W20260806 22:52:23.837178 13301 heartbeater.cc:644] Failed to heartbeat to 127.28.218.254:45957 (0 consecutive failures): Network error: Failed to ping master at 127.28.218.254:45957: Client connection negotiation failed: client connection to 127.28.218.254:45957: connect: Connection refused (error 111)
I20260806 22:52:23.837442 13303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:23.837354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:45957 (local address 127.28.218.195:40173)
0806 22:52:23.837399 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0806 22:52:23.837424 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:45957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260806 22:52:23.837422 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 13165
I20260806 22:52:23.837745 13303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:23.837650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:45957 (local address 127.28.218.195:56201)
0806 22:52:23.837696 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0806 22:52:23.837722 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:45957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260806 22:52:23.837878 13301 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 22:52:24.608507 13305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:24.608274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:45957 (local address 127.28.218.193:51531)
0806 22:52:24.608390 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0806 22:52:24.608422 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:45957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":59,"threads_started":1}
I20260806 22:52:24.700501 13306 raft_consensus.cc:482] T b2984fd22aef4f5885be76bbae247bc9 P 7a4f5f00a15646a3ae17db2711fff3e0 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:52:24.700639 13306 raft_consensus.cc:504] T b2984fd22aef4f5885be76bbae247bc9 P 7a4f5f00a15646a3ae17db2711fff3e0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4233903adc864154b6565c00f5654b70" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 45531 } } peers { permanent_uuid: "860a5cffbfbc4199a28144ec07d1fcd3" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 42589 } } peers { permanent_uuid: "7a4f5f00a15646a3ae17db2711fff3e0" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 34809 } }
I20260806 22:52:24.701002 13306 leader_election.cc:288] T b2984fd22aef4f5885be76bbae247bc9 P 7a4f5f00a15646a3ae17db2711fff3e0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4233903adc864154b6565c00f5654b70 (127.28.218.195:45531), 860a5cffbfbc4199a28144ec07d1fcd3 (127.28.218.194:42589)
Aug 06 22:52:24 dist-test-slave-1xrh krb5kdc[11659](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056743, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.193@KRBTEST.COM for kudu/127.28.218.195@KRBTEST.COM
Aug 06 22:52:24 dist-test-slave-1xrh krb5kdc[11659](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056743, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.193@KRBTEST.COM for kudu/127.28.218.194@KRBTEST.COM
I20260806 22:52:24.711004 13305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:24.701166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.195:45531 (local address 127.28.218.193:37491)
0806 22:52:24.701214 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0806 22:52:24.701226 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:52:24.701282 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:52:24.702092 (+ 810us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:52:24.702095 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:52:24.702471 (+ 376us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:52:24.702645 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:52:24.702651 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:52:24.703914 (+ 1263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:52:24.703915 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:52:24.704601 (+ 686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:52:24.704604 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:52:24.705181 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:52:24.705182 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:52:24.705304 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:52:24.705959 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:52:24.705964 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:52:24.707517 (+ 1553us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:52:24.709418 (+ 1901us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:52:24.709424 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:52:24.709426 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:52:24.709644 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:52:24.709868 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:52:24.709869 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:52:24.709870 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:52:24.709989 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:52:24.710515 (+ 526us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:52:24.710529 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:52:24.710680 (+ 151us) client_negotiation.cc:769] Sending connection context
0806 22:52:24.710792 (+ 112us) client_negotiation.cc:240] Negotiation successful
0806 22:52:24.710940 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29,"mutex_wait_us":42}
I20260806 22:52:24.711496 13307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:24.701269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.193:37491 (local address 127.28.218.195:45531)
0806 22:52:24.701430 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0806 22:52:24.701440 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:52:24.701471 (+ 31us) server_negotiation.cc:407] Connection header received
0806 22:52:24.701862 (+ 391us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:52:24.701864 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:52:24.701913 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:52:24.702020 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:52:24.702761 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:24.703815 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:24.704726 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:24.705090 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:24.705202 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:52:24.707625 (+ 2423us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:52:24.707628 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:52:24.707629 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:52:24.707637 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:52:24.709294 (+ 1657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:52:24.709720 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:52:24.709722 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:52:24.709722 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:52:24.709798 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:52:24.710081 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:52:24.710082 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:52:24.710082 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:52:24.710408 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:52:24.710496 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 22:52:24.711224 (+ 728us) server_negotiation.cc:299] Negotiation successful
0806 22:52:24.711369 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":56,"threads_started":1}
I20260806 22:52:24.711637 13308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:24.701308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.194:42589 (local address 127.28.218.193:43927)
0806 22:52:24.701427 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0806 22:52:24.701438 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:52:24.701499 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:52:24.702192 (+ 693us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:52:24.702193 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:52:24.702491 (+ 298us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:52:24.702645 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:52:24.702650 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:52:24.703798 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:52:24.703809 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:52:24.704600 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:52:24.704603 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:52:24.705179 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:52:24.705180 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:52:24.705300 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:52:24.705905 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:52:24.705915 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:52:24.708034 (+ 2119us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:52:24.710265 (+ 2231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:52:24.710266 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:52:24.710266 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:52:24.710483 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:52:24.710729 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:52:24.710730 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:52:24.710730 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:52:24.710794 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:52:24.711351 (+ 557us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:52:24.711352 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:52:24.711392 (+ 40us) client_negotiation.cc:769] Sending connection context
0806 22:52:24.711461 (+ 69us) client_negotiation.cc:240] Negotiation successful
0806 22:52:24.711547 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":54,"threads_started":1}
I20260806 22:52:24.711751 13309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:24.701379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.193:43927 (local address 127.28.218.194:42589)
0806 22:52:24.701540 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0806 22:52:24.701551 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 22:52:24.701571 (+ 20us) server_negotiation.cc:407] Connection header received
0806 22:52:24.701975 (+ 404us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:52:24.701977 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:52:24.702072 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:52:24.702205 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:52:24.702761 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:24.703708 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:24.704687 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:24.705088 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:24.705204 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:52:24.708156 (+ 2952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:52:24.708160 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:52:24.708160 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:52:24.708172 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:52:24.710177 (+ 2005us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:52:24.710570 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:52:24.710572 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:52:24.710572 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:52:24.710649 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:52:24.710856 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:52:24.710857 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:52:24.710857 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:52:24.711270 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:52:24.711350 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 22:52:24.711516 (+ 166us) server_negotiation.cc:299] Negotiation successful
0806 22:52:24.711655 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":65,"threads_started":1}
W20260806 22:52:24.712203 12905 leader_election.cc:341] T b2984fd22aef4f5885be76bbae247bc9 P 7a4f5f00a15646a3ae17db2711fff3e0 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4233903adc864154b6565c00f5654b70 (127.28.218.195:45531): Not found: Tablet not found: b2984fd22aef4f5885be76bbae247bc9
W20260806 22:52:24.712340 12905 leader_election.cc:341] T b2984fd22aef4f5885be76bbae247bc9 P 7a4f5f00a15646a3ae17db2711fff3e0 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 860a5cffbfbc4199a28144ec07d1fcd3 (127.28.218.194:42589): Not found: Tablet not found: b2984fd22aef4f5885be76bbae247bc9
I20260806 22:52:24.712383 12905 leader_election.cc:302] T b2984fd22aef4f5885be76bbae247bc9 P 7a4f5f00a15646a3ae17db2711fff3e0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a4f5f00a15646a3ae17db2711fff3e0; no voters: 4233903adc864154b6565c00f5654b70, 860a5cffbfbc4199a28144ec07d1fcd3
I20260806 22:52:24.712534 13306 raft_consensus.cc:2726] T b2984fd22aef4f5885be76bbae247bc9 P 7a4f5f00a15646a3ae17db2711fff3e0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 22:52:24.712590 13310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:24.712427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:45957 (local address 127.28.218.194:41233)
0806 22:52:24.712522 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0806 22:52:24.712547 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:45957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80,"thread_start_us":41,"threads_started":1}
I20260806 22:52:24.838470 13311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:24.838167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:45957 (local address 127.28.218.195:33817)
0806 22:52:24.838402 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0806 22:52:24.838439 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:45957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":64,"threads_started":1}
I20260806 22:52:25.609288 13312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:25.609073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:45957 (local address 127.28.218.193:49331)
0806 22:52:25.609211 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0806 22:52:25.609244 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:45957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":81,"threads_started":1}
I20260806 22:52:25.713129 13313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:25.712924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:45957 (local address 127.28.218.194:40329)
0806 22:52:25.713058 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0806 22:52:25.713094 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:45957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":74,"threads_started":1}
I20260806 22:52:25.839195 13314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:25.838974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:45957 (local address 127.28.218.195:48023)
0806 22:52:25.839097 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0806 22:52:25.839146 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:45957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":69,"threads_started":1}
I20260806 22:52:26.549646 13315 raft_consensus.cc:482] T b2984fd22aef4f5885be76bbae247bc9 P 7a4f5f00a15646a3ae17db2711fff3e0 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:52:26.549758 13315 raft_consensus.cc:504] T b2984fd22aef4f5885be76bbae247bc9 P 7a4f5f00a15646a3ae17db2711fff3e0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4233903adc864154b6565c00f5654b70" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 45531 } } peers { permanent_uuid: "860a5cffbfbc4199a28144ec07d1fcd3" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 42589 } } peers { permanent_uuid: "7a4f5f00a15646a3ae17db2711fff3e0" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 34809 } }
I20260806 22:52:26.549939 13315 leader_election.cc:288] T b2984fd22aef4f5885be76bbae247bc9 P 7a4f5f00a15646a3ae17db2711fff3e0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4233903adc864154b6565c00f5654b70 (127.28.218.195:45531), 860a5cffbfbc4199a28144ec07d1fcd3 (127.28.218.194:42589)
W20260806 22:52:26.550387 12905 leader_election.cc:341] T b2984fd22aef4f5885be76bbae247bc9 P 7a4f5f00a15646a3ae17db2711fff3e0 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 860a5cffbfbc4199a28144ec07d1fcd3 (127.28.218.194:42589): Not found: Tablet not found: b2984fd22aef4f5885be76bbae247bc9
W20260806 22:52:26.550465 12905 leader_election.cc:341] T b2984fd22aef4f5885be76bbae247bc9 P 7a4f5f00a15646a3ae17db2711fff3e0 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4233903adc864154b6565c00f5654b70 (127.28.218.195:45531): Not found: Tablet not found: b2984fd22aef4f5885be76bbae247bc9
I20260806 22:52:26.550483 12905 leader_election.cc:302] T b2984fd22aef4f5885be76bbae247bc9 P 7a4f5f00a15646a3ae17db2711fff3e0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a4f5f00a15646a3ae17db2711fff3e0; no voters: 4233903adc864154b6565c00f5654b70, 860a5cffbfbc4199a28144ec07d1fcd3
I20260806 22:52:26.550590 13315 raft_consensus.cc:2726] T b2984fd22aef4f5885be76bbae247bc9 P 7a4f5f00a15646a3ae17db2711fff3e0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 22:52:26.609979 13316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:26.609746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:45957 (local address 127.28.218.193:34101)
0806 22:52:26.609894 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0806 22:52:26.609930 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:45957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":76,"threads_started":1}
I20260806 22:52:26.713840 13317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:26.713646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:45957 (local address 127.28.218.194:51521)
0806 22:52:26.713764 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0806 22:52:26.713799 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:45957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":55,"threads_started":1}
I20260806 22:52:26.839802 13318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:26.839602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:45957 (local address 127.28.218.195:40071)
0806 22:52:26.839728 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0806 22:52:26.839765 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:45957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":72,"threads_started":1}
I20260806 22:52:27.610730 13319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:27.610517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:45957 (local address 127.28.218.193:49019)
0806 22:52:27.610650 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0806 22:52:27.610685 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:45957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":61,"threads_started":1}
I20260806 22:52:27.714532 13320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:27.714314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:45957 (local address 127.28.218.194:56705)
0806 22:52:27.714463 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0806 22:52:27.714497 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:45957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":88,"threads_started":1}
I20260806 22:52:27.840413 13321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:27.840204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:45957 (local address 127.28.218.195:34433)
0806 22:52:27.840342 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0806 22:52:27.840377 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:45957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":57,"threads_started":1}
I20260806 22:52:28.611541 13322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:28.611253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:45957 (local address 127.28.218.193:52263)
0806 22:52:28.611438 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0806 22:52:28.611476 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:45957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":85,"threads_started":1}
I20260806 22:52:28.665117 13323 raft_consensus.cc:482] T b2984fd22aef4f5885be76bbae247bc9 P 7a4f5f00a15646a3ae17db2711fff3e0 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:52:28.665206 13323 raft_consensus.cc:504] T b2984fd22aef4f5885be76bbae247bc9 P 7a4f5f00a15646a3ae17db2711fff3e0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4233903adc864154b6565c00f5654b70" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 45531 } } peers { permanent_uuid: "860a5cffbfbc4199a28144ec07d1fcd3" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 42589 } } peers { permanent_uuid: "7a4f5f00a15646a3ae17db2711fff3e0" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 34809 } }
I20260806 22:52:28.665354 13323 leader_election.cc:288] T b2984fd22aef4f5885be76bbae247bc9 P 7a4f5f00a15646a3ae17db2711fff3e0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4233903adc864154b6565c00f5654b70 (127.28.218.195:45531), 860a5cffbfbc4199a28144ec07d1fcd3 (127.28.218.194:42589)
W20260806 22:52:28.665704 12905 leader_election.cc:341] T b2984fd22aef4f5885be76bbae247bc9 P 7a4f5f00a15646a3ae17db2711fff3e0 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 860a5cffbfbc4199a28144ec07d1fcd3 (127.28.218.194:42589): Not found: Tablet not found: b2984fd22aef4f5885be76bbae247bc9
W20260806 22:52:28.665799 12905 leader_election.cc:341] T b2984fd22aef4f5885be76bbae247bc9 P 7a4f5f00a15646a3ae17db2711fff3e0 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4233903adc864154b6565c00f5654b70 (127.28.218.195:45531): Not found: Tablet not found: b2984fd22aef4f5885be76bbae247bc9
I20260806 22:52:28.665815 12905 leader_election.cc:302] T b2984fd22aef4f5885be76bbae247bc9 P 7a4f5f00a15646a3ae17db2711fff3e0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a4f5f00a15646a3ae17db2711fff3e0; no voters: 4233903adc864154b6565c00f5654b70, 860a5cffbfbc4199a28144ec07d1fcd3
I20260806 22:52:28.665880 13323 raft_consensus.cc:2726] T b2984fd22aef4f5885be76bbae247bc9 P 7a4f5f00a15646a3ae17db2711fff3e0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 22:52:28.715325 13324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:28.715081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:45957 (local address 127.28.218.194:42193)
0806 22:52:28.715242 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0806 22:52:28.715274 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:45957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":109,"threads_started":1}
I20260806 22:52:28.841118 13325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:28.840893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:45957 (local address 127.28.218.195:36983)
0806 22:52:28.841048 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0806 22:52:28.841081 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:45957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":87,"threads_started":1}
I20260806 22:52:29.612175 13326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:29.611957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:45957 (local address 127.28.218.193:41215)
0806 22:52:29.612095 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0806 22:52:29.612128 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:45957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":77,"threads_started":1}
I20260806 22:52:29.715915 13327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:29.715704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:45957 (local address 127.28.218.194:34555)
0806 22:52:29.715839 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0806 22:52:29.715876 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:45957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":60,"threads_started":1}
I20260806 22:52:29.841832 13328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:29.841614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:45957 (local address 127.28.218.195:40621)
0806 22:52:29.841765 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0806 22:52:29.841795 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:45957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":87,"threads_started":1}
I20260806 22:52:30.364050 13329 raft_consensus.cc:482] T b2984fd22aef4f5885be76bbae247bc9 P 7a4f5f00a15646a3ae17db2711fff3e0 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:52:30.364167 13329 raft_consensus.cc:504] T b2984fd22aef4f5885be76bbae247bc9 P 7a4f5f00a15646a3ae17db2711fff3e0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4233903adc864154b6565c00f5654b70" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 45531 } } peers { permanent_uuid: "860a5cffbfbc4199a28144ec07d1fcd3" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 42589 } } peers { permanent_uuid: "7a4f5f00a15646a3ae17db2711fff3e0" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 34809 } }
I20260806 22:52:30.364373 13329 leader_election.cc:288] T b2984fd22aef4f5885be76bbae247bc9 P 7a4f5f00a15646a3ae17db2711fff3e0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4233903adc864154b6565c00f5654b70 (127.28.218.195:45531), 860a5cffbfbc4199a28144ec07d1fcd3 (127.28.218.194:42589)
W20260806 22:52:30.364763 12905 leader_election.cc:341] T b2984fd22aef4f5885be76bbae247bc9 P 7a4f5f00a15646a3ae17db2711fff3e0 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4233903adc864154b6565c00f5654b70 (127.28.218.195:45531): Not found: Tablet not found: b2984fd22aef4f5885be76bbae247bc9
W20260806 22:52:30.364833 12905 leader_election.cc:341] T b2984fd22aef4f5885be76bbae247bc9 P 7a4f5f00a15646a3ae17db2711fff3e0 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 860a5cffbfbc4199a28144ec07d1fcd3 (127.28.218.194:42589): Not found: Tablet not found: b2984fd22aef4f5885be76bbae247bc9
I20260806 22:52:30.364876 12905 leader_election.cc:302] T b2984fd22aef4f5885be76bbae247bc9 P 7a4f5f00a15646a3ae17db2711fff3e0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a4f5f00a15646a3ae17db2711fff3e0; no voters: 4233903adc864154b6565c00f5654b70, 860a5cffbfbc4199a28144ec07d1fcd3
I20260806 22:52:30.364949 13329 raft_consensus.cc:2726] T b2984fd22aef4f5885be76bbae247bc9 P 7a4f5f00a15646a3ae17db2711fff3e0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 22:52:30.612882 13331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:30.612602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:45957 (local address 127.28.218.193:51419)
0806 22:52:30.612799 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0806 22:52:30.612835 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:45957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":59,"threads_started":1}
I20260806 22:52:30.716507 13332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:30.716328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:45957 (local address 127.28.218.194:59939)
0806 22:52:30.716439 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0806 22:52:30.716470 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:45957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":55,"threads_started":1}
I20260806 22:52:30.842535 13333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:30.842334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:45957 (local address 127.28.218.195:46185)
0806 22:52:30.842466 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0806 22:52:30.842499 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:45957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":75,"threads_started":1}
I20260806 22:52:31.613559 13334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:31.613349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:45957 (local address 127.28.218.193:55241)
0806 22:52:31.613479 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0806 22:52:31.613512 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:45957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":61,"threads_started":1}
I20260806 22:52:31.717175 13335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:31.716968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:45957 (local address 127.28.218.194:41893)
0806 22:52:31.717102 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0806 22:52:31.717135 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:45957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":77,"threads_started":1}
I20260806 22:52:31.843326 13336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:31.842943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:45957 (local address 127.28.218.195:39689)
0806 22:52:31.843231 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0806 22:52:31.843274 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:45957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":69,"threads_started":1}
I20260806 22:52:32.614204 13337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:32.614012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:45957 (local address 127.28.218.193:36189)
0806 22:52:32.614124 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0806 22:52:32.614157 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:45957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":60,"threads_started":1}
I20260806 22:52:32.717783 13338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:32.717562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:45957 (local address 127.28.218.194:53003)
0806 22:52:32.717707 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0806 22:52:32.717746 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:45957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":71,"threads_started":1}
I20260806 22:52:32.843933 13339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:32.843739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:45957 (local address 127.28.218.195:54077)
0806 22:52:32.843860 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0806 22:52:32.843892 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:45957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":73,"threads_started":1}
I20260806 22:52:33.084028 13340 raft_consensus.cc:482] T b2984fd22aef4f5885be76bbae247bc9 P 7a4f5f00a15646a3ae17db2711fff3e0 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:52:33.084126 13340 raft_consensus.cc:504] T b2984fd22aef4f5885be76bbae247bc9 P 7a4f5f00a15646a3ae17db2711fff3e0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4233903adc864154b6565c00f5654b70" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 45531 } } peers { permanent_uuid: "860a5cffbfbc4199a28144ec07d1fcd3" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 42589 } } peers { permanent_uuid: "7a4f5f00a15646a3ae17db2711fff3e0" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 34809 } }
I20260806 22:52:33.084290 13340 leader_election.cc:288] T b2984fd22aef4f5885be76bbae247bc9 P 7a4f5f00a15646a3ae17db2711fff3e0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4233903adc864154b6565c00f5654b70 (127.28.218.195:45531), 860a5cffbfbc4199a28144ec07d1fcd3 (127.28.218.194:42589)
W20260806 22:52:33.084682 12905 leader_election.cc:341] T b2984fd22aef4f5885be76bbae247bc9 P 7a4f5f00a15646a3ae17db2711fff3e0 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 860a5cffbfbc4199a28144ec07d1fcd3 (127.28.218.194:42589): Not found: Tablet not found: b2984fd22aef4f5885be76bbae247bc9
W20260806 22:52:33.084743 12905 leader_election.cc:341] T b2984fd22aef4f5885be76bbae247bc9 P 7a4f5f00a15646a3ae17db2711fff3e0 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4233903adc864154b6565c00f5654b70 (127.28.218.195:45531): Not found: Tablet not found: b2984fd22aef4f5885be76bbae247bc9
I20260806 22:52:33.084758 12905 leader_election.cc:302] T b2984fd22aef4f5885be76bbae247bc9 P 7a4f5f00a15646a3ae17db2711fff3e0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a4f5f00a15646a3ae17db2711fff3e0; no voters: 4233903adc864154b6565c00f5654b70, 860a5cffbfbc4199a28144ec07d1fcd3
I20260806 22:52:33.084824 13340 raft_consensus.cc:2726] T b2984fd22aef4f5885be76bbae247bc9 P 7a4f5f00a15646a3ae17db2711fff3e0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 22:52:33.614850 13341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:33.614627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:45957 (local address 127.28.218.193:56479)
0806 22:52:33.614772 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0806 22:52:33.614801 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:45957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":71,"threads_started":1}
I20260806 22:52:33.718465 13342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:33.718239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:45957 (local address 127.28.218.194:59669)
0806 22:52:33.718356 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0806 22:52:33.718387 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:45957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":55,"threads_started":1}
I20260806 22:52:33.844538 13343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:33.844324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:45957 (local address 127.28.218.195:47431)
0806 22:52:33.844467 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0806 22:52:33.844498 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:45957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":82,"threads_started":1}
I20260806 22:52:34.615536 13344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:34.615330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:45957 (local address 127.28.218.193:40395)
0806 22:52:34.615447 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0806 22:52:34.615487 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:45957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":60,"threads_started":1}
I20260806 22:52:34.719059 13345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:34.718866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:45957 (local address 127.28.218.194:52385)
0806 22:52:34.718990 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0806 22:52:34.719020 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:45957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":72,"threads_started":1}
I20260806 22:52:34.845285 13346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:34.844915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:45957 (local address 127.28.218.195:37637)
0806 22:52:34.845210 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0806 22:52:34.845246 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:45957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":66,"threads_started":1}
I20260806 22:52:35.616263 13347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:35.616001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:45957 (local address 127.28.218.193:36245)
0806 22:52:35.616136 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0806 22:52:35.616184 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:45957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":65,"threads_started":1}
I20260806 22:52:35.719640 13348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:35.719440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:45957 (local address 127.28.218.194:58651)
0806 22:52:35.719561 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0806 22:52:35.719597 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:45957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":63,"threads_started":1}
I20260806 22:52:35.846076 13349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:35.845843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:45957 (local address 127.28.218.195:37453)
0806 22:52:35.846004 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0806 22:52:35.846040 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:45957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":65,"threads_started":1}
I20260806 22:52:36.616899 13350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:36.616660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:45957 (local address 127.28.218.193:33229)
0806 22:52:36.616816 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0806 22:52:36.616852 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:45957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":81,"threads_started":1}
I20260806 22:52:36.720234 13351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:36.720043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:45957 (local address 127.28.218.194:41733)
0806 22:52:36.720158 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0806 22:52:36.720192 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:45957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":58,"threads_started":1}
I20260806 22:52:36.846689 13352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:36.846474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:45957 (local address 127.28.218.195:57873)
0806 22:52:36.846610 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0806 22:52:36.846647 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:45957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":58,"threads_started":1}
I20260806 22:52:37.617625 13353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:37.617379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:45957 (local address 127.28.218.193:60173)
0806 22:52:37.617509 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0806 22:52:37.617577 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:45957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":64,"threads_started":1}
I20260806 22:52:37.720836 13354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:37.720649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:45957 (local address 127.28.218.194:46197)
0806 22:52:37.720764 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0806 22:52:37.720797 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:45957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":63,"threads_started":1}
I20260806 22:52:37.847476 13355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:37.847227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:45957 (local address 127.28.218.195:35709)
0806 22:52:37.847392 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0806 22:52:37.847435 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:45957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":78,"threads_started":1}
I20260806 22:52:38.618363 13357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:38.618173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:45957 (local address 127.28.218.193:34497)
0806 22:52:38.618288 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0806 22:52:38.618320 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:45957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":59,"threads_started":1}
I20260806 22:52:38.721495 13358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:38.721291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:45957 (local address 127.28.218.194:51827)
0806 22:52:38.721414 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0806 22:52:38.721444 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:45957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":61,"threads_started":1}
I20260806 22:52:38.840013 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 12859
I20260806 22:52:38.846683 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 13028
I20260806 22:52:38.848546 13359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:38.848364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:45957 (local address 127.28.218.195:47797)
0806 22:52:38.848478 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0806 22:52:38.848509 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:45957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":62,"threads_started":1}
I20260806 22:52:38.851495 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 13165
2026-08-06T22:52:38Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18587 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:52:38 dist-test-slave-1xrh krb5kdc[13362](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:52:38 dist-test-slave-1xrh krb5kdc[13362](info): set up 2 sockets
Aug 06 22:52:38 dist-test-slave-1xrh krb5kdc[13362](info): commencing operation
krb5kdc: starting...
W20260806 22:52:39.898046 29547 mini_kdc.cc:121] Time spent starting KDC: real 1.026s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/krb5kdc/test-user.keytab.
Aug 06 22:52:39 dist-test-slave-1xrh krb5kdc[13362](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056759, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:52:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:52:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.218.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.218.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.218.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab.
Entry for principal kudu/127.28.218.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab.
I20260806 22:52:40.020373 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.218.254:35241
--webserver_interface=127.28.218.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab
--principal=kudu/127.28.218.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.218.212:40995
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.218.254:35241
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:52:40.101361 13376 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:52:40.101585 13376 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:52:40.101621 13376 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:52:40.102764 13376 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 22:52:40.102830 13376 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 22:52:40.102854 13376 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 22:52:40.102871 13376 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:52:40.102905 13376 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:52:40.102929 13376 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:52:40.102967 13376 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:52:40.102996 13376 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:52:40.104537 13376 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:40995
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.28.218.254:35241
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab
--principal=kudu/127.28.218.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.218.254:35241
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.218.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13376
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:52:40.104806 13376 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:52:40.105031 13376 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:52:40.109763 13383 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:52:40.109769 13382 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:52:40.109992 13376 server_base.cc:1034] running on GCE node
W20260806 22:52:40.109769 13385 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:52:40.110297 13376 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:52:40.110620 13376 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:52:40.111786 13376 hybrid_clock.cc:648] HybridClock initialized: now 1786056760111772 us; error 44 us; skew 500 ppm
Aug 06 22:52:40 dist-test-slave-1xrh krb5kdc[13362](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056760, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:52:40.114233 13376 init.cc:377] Logged in from keytab as kudu/127.28.218.254@KRBTEST.COM (short username kudu)
I20260806 22:52:40.114639 13376 webserver.cc:460] Webserver started at http://127.28.218.254:35267/ using document root <none> and password file <none>
I20260806 22:52:40.114885 13376 fs_manager.cc:359] Metadata directory not provided
I20260806 22:52:40.114929 13376 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:52:40.115015 13376 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:52:40.116024 13376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/master-0/data/instance:
uuid: "97cb235f35f847ba97cf43d0dd94c0b5"
format_stamp: "Formatted at 2026-08-06 22:52:40 on dist-test-slave-1xrh"
server_key: "0b6344b64d49e81ccfbc16cbdecc3682"
server_key_iv: "8dff30e4a2f249b76ea801a7289e364f"
server_key_version: "encryptionkey@0"
I20260806 22:52:40.116336 13376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/master-0/wal/instance:
uuid: "97cb235f35f847ba97cf43d0dd94c0b5"
format_stamp: "Formatted at 2026-08-06 22:52:40 on dist-test-slave-1xrh"
server_key: "0b6344b64d49e81ccfbc16cbdecc3682"
server_key_iv: "8dff30e4a2f249b76ea801a7289e364f"
server_key_version: "encryptionkey@0"
I20260806 22:52:40.117576 13376 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:52:40.118257 13393 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:52:40.118448 13376 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:52:40.118537 13376 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/master-0/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/master-0/wal
uuid: "97cb235f35f847ba97cf43d0dd94c0b5"
format_stamp: "Formatted at 2026-08-06 22:52:40 on dist-test-slave-1xrh"
server_key: "0b6344b64d49e81ccfbc16cbdecc3682"
server_key_iv: "8dff30e4a2f249b76ea801a7289e364f"
server_key_version: "encryptionkey@0"
I20260806 22:52:40.118614 13376 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:52:40.133291 13376 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:52:40.135375 13376 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:52:40.135527 13376 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:52:40.139546 13376 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.254:35241
I20260806 22:52:40.139590 13446 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.254:35241 every 8 connection(s)
I20260806 22:52:40.139919 13376 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
I20260806 22:52:40.140702 13447 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:52:40.142818 13447 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97cb235f35f847ba97cf43d0dd94c0b5: Bootstrap starting.
I20260806 22:52:40.143397 13447 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 97cb235f35f847ba97cf43d0dd94c0b5: Neither blocks nor log segments found. Creating new log.
I20260806 22:52:40.143690 13447 log.cc:824] T 00000000000000000000000000000000 P 97cb235f35f847ba97cf43d0dd94c0b5: Log is configured to *not* fsync() on all Append() calls
I20260806 22:52:40.144374 13447 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97cb235f35f847ba97cf43d0dd94c0b5: No bootstrap required, opened a new log
I20260806 22:52:40.145236 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 13376
I20260806 22:52:40.145380 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/master-0/wal/instance
I20260806 22:52:40.145630 29547 external_mini_cluster.cc:1428] Setting key 21496e9c6763c236e5963ce1f4e61ca8
I20260806 22:52:40.146370 13447 raft_consensus.cc:348] T 00000000000000000000000000000000 P 97cb235f35f847ba97cf43d0dd94c0b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97cb235f35f847ba97cf43d0dd94c0b5" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 35241 } }
I20260806 22:52:40.146543 13447 raft_consensus.cc:374] T 00000000000000000000000000000000 P 97cb235f35f847ba97cf43d0dd94c0b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:52:40.146581 13447 raft_consensus.cc:729] T 00000000000000000000000000000000 P 97cb235f35f847ba97cf43d0dd94c0b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97cb235f35f847ba97cf43d0dd94c0b5, State: Initialized, Role: FOLLOWER
I20260806 22:52:40.146698 13447 consensus_queue.cc:260] T 00000000000000000000000000000000 P 97cb235f35f847ba97cf43d0dd94c0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97cb235f35f847ba97cf43d0dd94c0b5" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 35241 } }
I20260806 22:52:40.146780 13447 raft_consensus.cc:388] T 00000000000000000000000000000000 P 97cb235f35f847ba97cf43d0dd94c0b5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:52:40.146811 13447 raft_consensus.cc:482] T 00000000000000000000000000000000 P 97cb235f35f847ba97cf43d0dd94c0b5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:52:40.146855 13447 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 97cb235f35f847ba97cf43d0dd94c0b5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:52:40.147891 13447 raft_consensus.cc:504] T 00000000000000000000000000000000 P 97cb235f35f847ba97cf43d0dd94c0b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97cb235f35f847ba97cf43d0dd94c0b5" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 35241 } }
I20260806 22:52:40.148010 13447 leader_election.cc:302] T 00000000000000000000000000000000 P 97cb235f35f847ba97cf43d0dd94c0b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 97cb235f35f847ba97cf43d0dd94c0b5; no voters:
I20260806 22:52:40.148156 13447 leader_election.cc:288] T 00000000000000000000000000000000 P 97cb235f35f847ba97cf43d0dd94c0b5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:52:40.148278 13452 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 97cb235f35f847ba97cf43d0dd94c0b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:52:40.148444 13447 sys_catalog.cc:564] T 00000000000000000000000000000000 P 97cb235f35f847ba97cf43d0dd94c0b5 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:52:40.148492 13452 raft_consensus.cc:686] T 00000000000000000000000000000000 P 97cb235f35f847ba97cf43d0dd94c0b5 [term 1 LEADER]: Becoming Leader. State: Replica: 97cb235f35f847ba97cf43d0dd94c0b5, State: Running, Role: LEADER
I20260806 22:52:40.148725 13452 consensus_queue.cc:237] T 00000000000000000000000000000000 P 97cb235f35f847ba97cf43d0dd94c0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97cb235f35f847ba97cf43d0dd94c0b5" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 35241 } }
I20260806 22:52:40.149144 13453 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97cb235f35f847ba97cf43d0dd94c0b5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "97cb235f35f847ba97cf43d0dd94c0b5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97cb235f35f847ba97cf43d0dd94c0b5" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 35241 } } }
I20260806 22:52:40.149151 13455 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97cb235f35f847ba97cf43d0dd94c0b5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 97cb235f35f847ba97cf43d0dd94c0b5. Latest consensus state: current_term: 1 leader_uuid: "97cb235f35f847ba97cf43d0dd94c0b5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97cb235f35f847ba97cf43d0dd94c0b5" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 35241 } } }
I20260806 22:52:40.149258 13453 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97cb235f35f847ba97cf43d0dd94c0b5 [sys.catalog]: This master's current role is: LEADER
I20260806 22:52:40.149263 13455 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97cb235f35f847ba97cf43d0dd94c0b5 [sys.catalog]: This master's current role is: LEADER
I20260806 22:52:40.149556 13462 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:52:40.150039 13462 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:52:40.151649 13462 catalog_manager.cc:1347] Generated new cluster ID: 3c7f71f6744b4458a6f069c02919cdd6
I20260806 22:52:40.151698 13462 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 22:52:40 dist-test-slave-1xrh krb5kdc[13362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056759, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.218.254@KRBTEST.COM
I20260806 22:52:40.157663 13451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:40.146154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56096 (local address 127.28.218.254:35241)
0806 22:52:40.146371 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0806 22:52:40.146380 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:52:40.146413 (+ 33us) server_negotiation.cc:407] Connection header received
0806 22:52:40.146530 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:52:40.146537 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:52:40.146592 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:52:40.146732 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:52:40.147531 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:40.148967 (+ 1436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:40.150083 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:40.150674 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:40.150800 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:52:40.153612 (+ 2812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:52:40.153616 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:52:40.153617 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:52:40.153627 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:52:40.155600 (+ 1973us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:52:40.156003 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:52:40.156006 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:52:40.156006 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:52:40.156115 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:52:40.156342 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:52:40.156343 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:52:40.156343 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:52:40.156644 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:52:40.156796 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0806 22:52:40.157292 (+ 496us) server_negotiation.cc:299] Negotiation successful
0806 22:52:40.157432 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":116,"threads_started":1}
I20260806 22:52:40.164206 13462 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:52:40.164719 13462 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:52:40.174244 13462 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 97cb235f35f847ba97cf43d0dd94c0b5: Generated new TSK 0
I20260806 22:52:40.174444 13462 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.28.218.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.218.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.218.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.193.keytab.
Entry for principal kudu/127.28.218.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.193.keytab.
I20260806 22:52:40.202360 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.218.193:0
--local_ip_for_outbound_sockets=127.28.218.193
--webserver_interface=127.28.218.193
--webserver_port=0
--tserver_master_addrs=127.28.218.254:35241
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.193.keytab
--principal=kudu/127.28.218.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.218.212:40995
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:52:40.283752 13475 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:52:40.283959 13475 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:52:40.283982 13475 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:52:40.285109 13475 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:52:40.285151 13475 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:52:40.285178 13475 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.218.193
I20260806 22:52:40.286631 13475 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:40995
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.193.keytab
--principal=kudu/127.28.218.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.218.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.218.193
--webserver_port=0
--tserver_master_addrs=127.28.218.254:35241
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13475
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.218.193
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:52:40.286867 13475 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:52:40.287063 13475 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:52:40.291965 13481 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:52:40.291977 13480 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:52:40.291971 13483 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:52:40.292142 13475 server_base.cc:1034] running on GCE node
I20260806 22:52:40.292448 13475 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:52:40.292721 13475 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:52:40.293859 13475 hybrid_clock.cc:648] HybridClock initialized: now 1786056760293822 us; error 51 us; skew 500 ppm
Aug 06 22:52:40 dist-test-slave-1xrh krb5kdc[13362](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056760, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:52:40.296329 13475 init.cc:377] Logged in from keytab as kudu/127.28.218.193@KRBTEST.COM (short username kudu)
I20260806 22:52:40.296808 13475 webserver.cc:460] Webserver started at http://127.28.218.193:35237/ using document root <none> and password file <none>
I20260806 22:52:40.297094 13475 fs_manager.cc:359] Metadata directory not provided
I20260806 22:52:40.297142 13475 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:52:40.297408 13475 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:52:40.298527 13475 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-0/data/instance:
uuid: "54b2b48fdddf4c6ea10ae001959e2ad3"
format_stamp: "Formatted at 2026-08-06 22:52:40 on dist-test-slave-1xrh"
server_key: "3a5eced6bd4526f6244d49e99c177928"
server_key_iv: "6dc9cd2527d76c346819058f2dfbdd57"
server_key_version: "encryptionkey@0"
I20260806 22:52:40.298887 13475 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-0/wal/instance:
uuid: "54b2b48fdddf4c6ea10ae001959e2ad3"
format_stamp: "Formatted at 2026-08-06 22:52:40 on dist-test-slave-1xrh"
server_key: "3a5eced6bd4526f6244d49e99c177928"
server_key_iv: "6dc9cd2527d76c346819058f2dfbdd57"
server_key_version: "encryptionkey@0"
I20260806 22:52:40.300252 13475 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:52:40.300951 13490 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:52:40.301126 13475 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:52:40.301223 13475 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-0/wal
uuid: "54b2b48fdddf4c6ea10ae001959e2ad3"
format_stamp: "Formatted at 2026-08-06 22:52:40 on dist-test-slave-1xrh"
server_key: "3a5eced6bd4526f6244d49e99c177928"
server_key_iv: "6dc9cd2527d76c346819058f2dfbdd57"
server_key_version: "encryptionkey@0"
I20260806 22:52:40.301309 13475 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:52:40.312274 13475 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:52:40.313977 13475 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:52:40.314128 13475 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:52:40.314343 13475 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:52:40.314632 13475 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:52:40.314685 13475 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:52:40.314726 13475 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:52:40.314769 13475 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:52:40.320729 13475 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.193:43611
I20260806 22:52:40.320775 13608 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.193:43611 every 8 connection(s)
I20260806 22:52:40.321069 13475 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-0/data/info.pb
Aug 06 22:52:40 dist-test-slave-1xrh krb5kdc[13362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056760, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.193@KRBTEST.COM for kudu/127.28.218.254@KRBTEST.COM
I20260806 22:52:40.327382 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 13475
I20260806 22:52:40.327489 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-0/wal/instance
I20260806 22:52:40.327728 29547 external_mini_cluster.cc:1428] Setting key 1074e4fc976f0cdc0e6763c3b63d5302
I20260806 22:52:40.330462 13451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:40.321775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.193:42487 (local address 127.28.218.254:35241)
0806 22:52:40.321849 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0806 22:52:40.321852 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:52:40.322028 (+ 176us) server_negotiation.cc:407] Connection header received
0806 22:52:40.322565 (+ 537us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:52:40.322566 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:52:40.322581 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:52:40.322634 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:52:40.323568 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:40.324069 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:40.325013 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:40.325200 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:40.325248 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:52:40.327523 (+ 2275us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:52:40.327525 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:52:40.327525 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:52:40.327534 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:52:40.329064 (+ 1530us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:52:40.329474 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:52:40.329475 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:52:40.329476 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:52:40.329538 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:52:40.329803 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:52:40.329804 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:52:40.329804 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:52:40.329921 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:52:40.329996 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 22:52:40.330277 (+ 281us) server_negotiation.cc:299] Negotiation successful
0806 22:52:40.330364 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 22:52:40.330626 13612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:40.321726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:35241 (local address 127.28.218.193:42487)
0806 22:52:40.321909 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0806 22:52:40.321944 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 22:52:40.322478 (+ 534us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:52:40.322682 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:52:40.322685 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:52:40.323306 (+ 621us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:52:40.323494 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:52:40.323499 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:52:40.324154 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:52:40.324155 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:52:40.324941 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:52:40.324943 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:52:40.325271 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:52:40.325272 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:52:40.325368 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:52:40.325849 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:52:40.325857 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:52:40.327414 (+ 1557us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:52:40.329149 (+ 1735us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:52:40.329156 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:52:40.329158 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:52:40.329382 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:52:40.329624 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:52:40.329625 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:52:40.329625 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:52:40.329730 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:52:40.329998 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:52:40.330006 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:52:40.330141 (+ 135us) client_negotiation.cc:769] Sending connection context
0806 22:52:40.330241 (+ 100us) client_negotiation.cc:240] Negotiation successful
0806 22:52:40.330395 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":101,"threads_started":1}
I20260806 22:52:40.331103 13610 heartbeater.cc:347] Connected to a master server at 127.28.218.254:35241
I20260806 22:52:40.331231 13610 heartbeater.cc:464] Registering TS with master...
I20260806 22:52:40.331401 13610 heartbeater.cc:511] Master 127.28.218.254:35241 requested a full tablet report, sending...
I20260806 22:52:40.331961 13411 ts_manager.cc:194] Registered new tserver with Master: 54b2b48fdddf4c6ea10ae001959e2ad3 (127.28.218.193:43611)
I20260806 22:52:40.332572 13411 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.218.193@KRBTEST.COM'} at 127.28.218.193:42487
WARNING: no policy specified for kudu/127.28.218.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.218.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.218.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.194.keytab.
Entry for principal kudu/127.28.218.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.194.keytab.
I20260806 22:52:40.354377 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.218.194:0
--local_ip_for_outbound_sockets=127.28.218.194
--webserver_interface=127.28.218.194
--webserver_port=0
--tserver_master_addrs=127.28.218.254:35241
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.194.keytab
--principal=kudu/127.28.218.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.218.212:40995
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:52:40.434650 13615 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:52:40.434883 13615 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:52:40.434912 13615 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:52:40.436007 13615 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:52:40.436041 13615 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:52:40.436066 13615 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.218.194
I20260806 22:52:40.437443 13615 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:40995
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.194.keytab
--principal=kudu/127.28.218.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.218.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.218.194
--webserver_port=0
--tserver_master_addrs=127.28.218.254:35241
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13615
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.218.194
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:52:40.437615 13615 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:52:40.437783 13615 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:52:40.442881 13621 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:52:40.442907 13623 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:52:40.442926 13620 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:52:40.443110 13615 server_base.cc:1034] running on GCE node
I20260806 22:52:40.443413 13615 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:52:40.443681 13615 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:52:40.444851 13615 hybrid_clock.cc:648] HybridClock initialized: now 1786056760444846 us; error 38 us; skew 500 ppm
Aug 06 22:52:40 dist-test-slave-1xrh krb5kdc[13362](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056760, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:52:40.447576 13615 init.cc:377] Logged in from keytab as kudu/127.28.218.194@KRBTEST.COM (short username kudu)
I20260806 22:52:40.448017 13615 webserver.cc:460] Webserver started at http://127.28.218.194:42403/ using document root <none> and password file <none>
I20260806 22:52:40.448299 13615 fs_manager.cc:359] Metadata directory not provided
I20260806 22:52:40.448377 13615 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:52:40.448532 13615 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:52:40.449688 13615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-1/data/instance:
uuid: "e9d32213de704457865ff3b95d803f34"
format_stamp: "Formatted at 2026-08-06 22:52:40 on dist-test-slave-1xrh"
server_key: "7de9e2e87fdbfc25d45f21e606f19973"
server_key_iv: "72954717981183c76dfab68b323d4e36"
server_key_version: "encryptionkey@0"
I20260806 22:52:40.450053 13615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-1/wal/instance:
uuid: "e9d32213de704457865ff3b95d803f34"
format_stamp: "Formatted at 2026-08-06 22:52:40 on dist-test-slave-1xrh"
server_key: "7de9e2e87fdbfc25d45f21e606f19973"
server_key_iv: "72954717981183c76dfab68b323d4e36"
server_key_version: "encryptionkey@0"
I20260806 22:52:40.451438 13615 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 22:52:40.452140 13631 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:52:40.452323 13615 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:52:40.452425 13615 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-1/wal
uuid: "e9d32213de704457865ff3b95d803f34"
format_stamp: "Formatted at 2026-08-06 22:52:40 on dist-test-slave-1xrh"
server_key: "7de9e2e87fdbfc25d45f21e606f19973"
server_key_iv: "72954717981183c76dfab68b323d4e36"
server_key_version: "encryptionkey@0"
I20260806 22:52:40.452502 13615 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:52:40.466607 13615 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:52:40.468708 13615 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:52:40.468873 13615 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:52:40.469141 13615 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:52:40.469460 13615 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:52:40.469517 13615 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:52:40.469559 13615 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:52:40.469588 13615 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:52:40.476040 13615 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.194:46483
I20260806 22:52:40.476138 13751 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.194:46483 every 8 connection(s)
I20260806 22:52:40.476502 13615 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-1/data/info.pb
I20260806 22:52:40.479354 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 13615
I20260806 22:52:40.479465 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-1/wal/instance
I20260806 22:52:40.479703 29547 external_mini_cluster.cc:1428] Setting key 57c3c8c255f1d60ffe750bcc2cdbb359
Aug 06 22:52:40 dist-test-slave-1xrh krb5kdc[13362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056760, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.194@KRBTEST.COM for kudu/127.28.218.254@KRBTEST.COM
I20260806 22:52:40.487283 13451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:40.477555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.194:58631 (local address 127.28.218.254:35241)
0806 22:52:40.477728 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0806 22:52:40.477731 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:52:40.477743 (+ 12us) server_negotiation.cc:407] Connection header received
0806 22:52:40.478283 (+ 540us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:52:40.478284 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:52:40.478295 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:52:40.478349 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:52:40.479274 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:40.480068 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:40.481455 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:40.481636 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:40.481689 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:52:40.484238 (+ 2549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:52:40.484239 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:52:40.484239 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:52:40.484245 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:52:40.485755 (+ 1510us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:52:40.486272 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:52:40.486272 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:52:40.486273 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:52:40.486320 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:52:40.486612 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:52:40.486613 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:52:40.486613 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:52:40.486710 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:52:40.486760 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 22:52:40.487104 (+ 344us) server_negotiation.cc:299] Negotiation successful
0806 22:52:40.487221 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260806 22:52:40.487531 13754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:40.477235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:35241 (local address 127.28.218.194:58631)
0806 22:52:40.477587 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0806 22:52:40.477632 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 22:52:40.478192 (+ 560us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:52:40.478399 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:52:40.478402 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:52:40.478952 (+ 550us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:52:40.479180 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:52:40.479186 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:52:40.480152 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:52:40.480153 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:52:40.481373 (+ 1220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:52:40.481376 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:52:40.481716 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:52:40.481717 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:52:40.481877 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:52:40.482467 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:52:40.482476 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:52:40.484131 (+ 1655us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:52:40.485849 (+ 1718us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:52:40.485857 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:52:40.485860 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:52:40.486192 (+ 332us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:52:40.486392 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:52:40.486393 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:52:40.486394 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:52:40.486541 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:52:40.486778 (+ 237us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:52:40.486785 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:52:40.486964 (+ 179us) client_negotiation.cc:769] Sending connection context
0806 22:52:40.487089 (+ 125us) client_negotiation.cc:240] Negotiation successful
0806 22:52:40.487286 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":78,"threads_started":1}
I20260806 22:52:40.488106 13752 heartbeater.cc:347] Connected to a master server at 127.28.218.254:35241
I20260806 22:52:40.488189 13752 heartbeater.cc:464] Registering TS with master...
I20260806 22:52:40.488396 13752 heartbeater.cc:511] Master 127.28.218.254:35241 requested a full tablet report, sending...
I20260806 22:52:40.488776 13411 ts_manager.cc:194] Registered new tserver with Master: e9d32213de704457865ff3b95d803f34 (127.28.218.194:46483)
I20260806 22:52:40.489161 13411 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.218.194@KRBTEST.COM'} at 127.28.218.194:58631
WARNING: no policy specified for kudu/127.28.218.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.218.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.218.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.195.keytab.
Entry for principal kudu/127.28.218.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.195.keytab.
I20260806 22:52:40.508399 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.218.195:0
--local_ip_for_outbound_sockets=127.28.218.195
--webserver_interface=127.28.218.195
--webserver_port=0
--tserver_master_addrs=127.28.218.254:35241
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.195.keytab
--principal=kudu/127.28.218.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.218.212:40995
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:52:40.588862 13757 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:52:40.589109 13757 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:52:40.589150 13757 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:52:40.590255 13757 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:52:40.590323 13757 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:52:40.590368 13757 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.218.195
I20260806 22:52:40.591890 13757 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:40995
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.195.keytab
--principal=kudu/127.28.218.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.218.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.218.195
--webserver_port=0
--tserver_master_addrs=127.28.218.254:35241
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13757
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.218.195
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:52:40.592163 13757 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:52:40.592386 13757 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:52:40.597509 13765 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:52:40.597503 13763 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:52:40.597519 13762 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:52:40.597570 13757 server_base.cc:1034] running on GCE node
I20260806 22:52:40.598126 13757 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:52:40.598400 13757 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:52:40.599550 13757 hybrid_clock.cc:648] HybridClock initialized: now 1786056760599545 us; error 39 us; skew 500 ppm
Aug 06 22:52:40 dist-test-slave-1xrh krb5kdc[13362](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056760, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:52:40.607060 13757 init.cc:377] Logged in from keytab as kudu/127.28.218.195@KRBTEST.COM (short username kudu)
I20260806 22:52:40.607532 13757 webserver.cc:460] Webserver started at http://127.28.218.195:44067/ using document root <none> and password file <none>
I20260806 22:52:40.607776 13757 fs_manager.cc:359] Metadata directory not provided
I20260806 22:52:40.607851 13757 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:52:40.608003 13757 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:52:40.609035 13757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-2/data/instance:
uuid: "60837b341f5d4b3eb7e6a53780b4abd3"
format_stamp: "Formatted at 2026-08-06 22:52:40 on dist-test-slave-1xrh"
server_key: "4b2e8dedab4651ff535f40daf801b17c"
server_key_iv: "dcc9a875e77dc9e6a9718036fd237485"
server_key_version: "encryptionkey@0"
I20260806 22:52:40.609426 13757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-2/wal/instance:
uuid: "60837b341f5d4b3eb7e6a53780b4abd3"
format_stamp: "Formatted at 2026-08-06 22:52:40 on dist-test-slave-1xrh"
server_key: "4b2e8dedab4651ff535f40daf801b17c"
server_key_iv: "dcc9a875e77dc9e6a9718036fd237485"
server_key_version: "encryptionkey@0"
I20260806 22:52:40.610788 13757 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:52:40.611635 13772 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:52:40.611789 13757 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 22:52:40.611896 13757 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-2/wal
uuid: "60837b341f5d4b3eb7e6a53780b4abd3"
format_stamp: "Formatted at 2026-08-06 22:52:40 on dist-test-slave-1xrh"
server_key: "4b2e8dedab4651ff535f40daf801b17c"
server_key_iv: "dcc9a875e77dc9e6a9718036fd237485"
server_key_version: "encryptionkey@0"
I20260806 22:52:40.611967 13757 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:52:40.636962 13757 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:52:40.638772 13757 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:52:40.638948 13757 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:52:40.639252 13757 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:52:40.639581 13757 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:52:40.639642 13757 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:52:40.639684 13757 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:52:40.639712 13757 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:52:40.645627 13757 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.195:38763
I20260806 22:52:40.645696 13892 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.195:38763 every 8 connection(s)
I20260806 22:52:40.646029 13757 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-2/data/info.pb
Aug 06 22:52:40 dist-test-slave-1xrh krb5kdc[13362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056760, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.195@KRBTEST.COM for kudu/127.28.218.254@KRBTEST.COM
I20260806 22:52:40.653656 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 13757
I20260806 22:52:40.653761 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-2/wal/instance
I20260806 22:52:40.653985 29547 external_mini_cluster.cc:1428] Setting key 6104a7c7816c7bd579756af0d22b9b56
I20260806 22:52:40.655857 13895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:40.646718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:35241 (local address 127.28.218.195:35513)
0806 22:52:40.646880 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0806 22:52:40.646920 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 22:52:40.647463 (+ 543us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:52:40.647671 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:52:40.647674 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:52:40.648282 (+ 608us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:52:40.648468 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:52:40.648472 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:52:40.649288 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:52:40.649289 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:52:40.650229 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:52:40.650232 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:52:40.650593 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:52:40.650594 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:52:40.650739 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:52:40.651229 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:52:40.651236 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:52:40.652560 (+ 1324us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:52:40.654299 (+ 1739us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:52:40.654320 (+ 21us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:52:40.654322 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:52:40.654680 (+ 358us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:52:40.654921 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:52:40.654922 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:52:40.654922 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:52:40.655085 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:52:40.655399 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:52:40.655416 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:52:40.655527 (+ 111us) client_negotiation.cc:769] Sending connection context
0806 22:52:40.655600 (+ 73us) client_negotiation.cc:240] Negotiation successful
0806 22:52:40.655709 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":96,"threads_started":1}
I20260806 22:52:40.655858 13451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:40.646745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.195:35513 (local address 127.28.218.254:35241)
0806 22:52:40.646811 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0806 22:52:40.646814 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:52:40.647020 (+ 206us) server_negotiation.cc:407] Connection header received
0806 22:52:40.647547 (+ 527us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:52:40.647548 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:52:40.647559 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:52:40.647621 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:52:40.648566 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:40.649139 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:40.650303 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:40.650491 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:40.650547 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:52:40.652644 (+ 2097us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:52:40.652645 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:52:40.652645 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:52:40.652651 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:52:40.654208 (+ 1557us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:52:40.654758 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:52:40.654758 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:52:40.654759 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:52:40.654847 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:52:40.655197 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:52:40.655198 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:52:40.655198 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:52:40.655324 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:52:40.655381 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 22:52:40.655685 (+ 304us) server_negotiation.cc:299] Negotiation successful
0806 22:52:40.655759 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 22:52:40.656296 13893 heartbeater.cc:347] Connected to a master server at 127.28.218.254:35241
I20260806 22:52:40.656386 13893 heartbeater.cc:464] Registering TS with master...
I20260806 22:52:40.656517 13893 heartbeater.cc:511] Master 127.28.218.254:35241 requested a full tablet report, sending...
I20260806 22:52:40.656862 13411 ts_manager.cc:194] Registered new tserver with Master: 60837b341f5d4b3eb7e6a53780b4abd3 (127.28.218.195:38763)
I20260806 22:52:40.657186 13411 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.218.195@KRBTEST.COM'} at 127.28.218.195:35513
I20260806 22:52:40.665876 29547 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:52:40.673432 13451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:40.667353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56108 (local address 127.28.218.254:35241)
0806 22:52:40.667422 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0806 22:52:40.667424 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:52:40.667522 (+ 98us) server_negotiation.cc:407] Connection header received
0806 22:52:40.667600 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:52:40.667601 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:52:40.667612 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:52:40.667669 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:52:40.668260 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:40.668728 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:40.669482 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:40.669681 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:40.669766 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:52:40.670942 (+ 1176us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:52:40.670943 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:52:40.670943 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:52:40.670951 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:52:40.672234 (+ 1283us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:52:40.672616 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:52:40.672617 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:52:40.672617 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:52:40.672678 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:52:40.672894 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:52:40.672894 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:52:40.672895 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:52:40.673038 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:52:40.673144 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0806 22:52:40.673261 (+ 117us) server_negotiation.cc:299] Negotiation successful
0806 22:52:40.673337 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 22:52:40.675091 13411 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56108:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:52:40.676626 13411 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:52:40.683174 13907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:40.679114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.193:43611 (local address 127.0.0.1:45724)
0806 22:52:40.679596 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0806 22:52:40.679605 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:52:40.679651 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:52:40.680035 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:52:40.680036 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:52:40.680038 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:52:40.680072 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:52:40.680073 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:52:40.681286 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:52:40.681287 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:52:40.682042 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:52:40.682044 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:52:40.682872 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:52:40.682873 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:52:40.682945 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:52:40.682959 (+ 14us) client_negotiation.cc:769] Sending connection context
0806 22:52:40.683034 (+ 75us) client_negotiation.cc:240] Negotiation successful
0806 22:52:40.683081 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":83,"threads_started":1}
I20260806 22:52:40.683498 13910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:40.679220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45724 (local address 127.28.218.193:43611)
0806 22:52:40.679394 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0806 22:52:40.679398 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:52:40.679690 (+ 292us) server_negotiation.cc:407] Connection header received
0806 22:52:40.679853 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:52:40.679856 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:52:40.679875 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:52:40.679954 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:52:40.680148 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:40.681190 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:40.682136 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:40.682786 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:40.682862 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:52:40.682914 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 22:52:40.683060 (+ 146us) server_negotiation.cc:299] Negotiation successful
0806 22:52:40.683102 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":105,"threads_started":1}
I20260806 22:52:40.683501 13905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:40.678909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.194:46483 (local address 127.0.0.1:37462)
0806 22:52:40.679066 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0806 22:52:40.679117 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0806 22:52:40.679197 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:52:40.680090 (+ 893us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:52:40.680091 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:52:40.680093 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:52:40.680146 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:52:40.680147 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:52:40.680983 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:52:40.680984 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:52:40.682271 (+ 1287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:52:40.682274 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:52:40.683270 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:52:40.683271 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:52:40.683335 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:52:40.683338 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:52:40.683388 (+ 50us) client_negotiation.cc:240] Negotiation successful
0806 22:52:40.683427 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"mutex_wait_us":39,"thread_start_us":42,"threads_started":1}
I20260806 22:52:40.683634 13908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:40.678995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37462 (local address 127.28.218.194:46483)
0806 22:52:40.679895 (+ 900us) server_negotiation.cc:206] Beginning negotiation
0806 22:52:40.679899 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:52:40.679908 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:52:40.679977 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:52:40.679979 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:52:40.679995 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:52:40.680090 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:52:40.680200 (+ 110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:40.680872 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:40.682354 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:40.683086 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:40.683172 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:52:40.683223 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 22:52:40.683421 (+ 198us) server_negotiation.cc:299] Negotiation successful
0806 22:52:40.683482 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":861,"thread_start_us":117,"threads_started":1}
I20260806 22:52:40.683671 13911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:40.679261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39848 (local address 127.28.218.195:38763)
0806 22:52:40.679440 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0806 22:52:40.679444 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:52:40.679514 (+ 70us) server_negotiation.cc:407] Connection header received
0806 22:52:40.679583 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:52:40.679586 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:52:40.679603 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:52:40.679685 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:52:40.679903 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:40.681004 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:40.682557 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:40.683182 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:40.683249 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:52:40.683302 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 22:52:40.683541 (+ 239us) server_negotiation.cc:299] Negotiation successful
0806 22:52:40.683583 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":104,"threads_started":1}
I20260806 22:52:40.683828 13909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:40.678909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.195:38763 (local address 127.0.0.1:39848)
0806 22:52:40.679448 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0806 22:52:40.679466 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 22:52:40.679518 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:52:40.679716 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:52:40.679719 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:52:40.679722 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:52:40.679785 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:52:40.679786 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:52:40.681107 (+ 1321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:52:40.681107 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:52:40.682480 (+ 1373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:52:40.682481 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:52:40.683338 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:52:40.683338 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:52:40.683417 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:52:40.683421 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:52:40.683467 (+ 46us) client_negotiation.cc:240] Negotiation successful
0806 22:52:40.683503 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":520,"mutex_wait_us":10,"thread_start_us":80,"threads_started":1}
I20260806 22:52:40.684444 13823 tablet_service.cc:1467] Processing CreateTablet for tablet 8ab758439aee4c089e718a92790961d5 (DEFAULT_TABLE table=test-table [id=b8c70b25d766460ebac7dc881afc300a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:52:40.684463 13682 tablet_service.cc:1467] Processing CreateTablet for tablet 8ab758439aee4c089e718a92790961d5 (DEFAULT_TABLE table=test-table [id=b8c70b25d766460ebac7dc881afc300a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:52:40.684710 13823 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ab758439aee4c089e718a92790961d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:52:40.684799 13682 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ab758439aee4c089e718a92790961d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:52:40.687274 13913 tablet_bootstrap.cc:492] T 8ab758439aee4c089e718a92790961d5 P e9d32213de704457865ff3b95d803f34: Bootstrap starting.
I20260806 22:52:40.687405 13540 tablet_service.cc:1467] Processing CreateTablet for tablet 8ab758439aee4c089e718a92790961d5 (DEFAULT_TABLE table=test-table [id=b8c70b25d766460ebac7dc881afc300a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:52:40.687700 13540 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ab758439aee4c089e718a92790961d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:52:40.687955 13913 tablet_bootstrap.cc:654] T 8ab758439aee4c089e718a92790961d5 P e9d32213de704457865ff3b95d803f34: Neither blocks nor log segments found. Creating new log.
I20260806 22:52:40.688448 13913 log.cc:824] T 8ab758439aee4c089e718a92790961d5 P e9d32213de704457865ff3b95d803f34: Log is configured to *not* fsync() on all Append() calls
I20260806 22:52:40.689145 13913 tablet_bootstrap.cc:492] T 8ab758439aee4c089e718a92790961d5 P e9d32213de704457865ff3b95d803f34: No bootstrap required, opened a new log
I20260806 22:52:40.689198 13913 ts_tablet_manager.cc:1397] T 8ab758439aee4c089e718a92790961d5 P e9d32213de704457865ff3b95d803f34: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:52:40.690249 13915 tablet_bootstrap.cc:492] T 8ab758439aee4c089e718a92790961d5 P 54b2b48fdddf4c6ea10ae001959e2ad3: Bootstrap starting.
I20260806 22:52:40.690325 13912 tablet_bootstrap.cc:492] T 8ab758439aee4c089e718a92790961d5 P 60837b341f5d4b3eb7e6a53780b4abd3: Bootstrap starting.
I20260806 22:52:40.690860 13915 tablet_bootstrap.cc:654] T 8ab758439aee4c089e718a92790961d5 P 54b2b48fdddf4c6ea10ae001959e2ad3: Neither blocks nor log segments found. Creating new log.
I20260806 22:52:40.690984 13912 tablet_bootstrap.cc:654] T 8ab758439aee4c089e718a92790961d5 P 60837b341f5d4b3eb7e6a53780b4abd3: Neither blocks nor log segments found. Creating new log.
I20260806 22:52:40.691032 13913 raft_consensus.cc:348] T 8ab758439aee4c089e718a92790961d5 P e9d32213de704457865ff3b95d803f34 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60837b341f5d4b3eb7e6a53780b4abd3" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 38763 } } peers { permanent_uuid: "e9d32213de704457865ff3b95d803f34" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 46483 } } peers { permanent_uuid: "54b2b48fdddf4c6ea10ae001959e2ad3" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 43611 } }
I20260806 22:52:40.691254 13913 raft_consensus.cc:374] T 8ab758439aee4c089e718a92790961d5 P e9d32213de704457865ff3b95d803f34 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:52:40.691308 13913 raft_consensus.cc:729] T 8ab758439aee4c089e718a92790961d5 P e9d32213de704457865ff3b95d803f34 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9d32213de704457865ff3b95d803f34, State: Initialized, Role: FOLLOWER
I20260806 22:52:40.691418 13913 consensus_queue.cc:260] T 8ab758439aee4c089e718a92790961d5 P e9d32213de704457865ff3b95d803f34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60837b341f5d4b3eb7e6a53780b4abd3" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 38763 } } peers { permanent_uuid: "e9d32213de704457865ff3b95d803f34" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 46483 } } peers { permanent_uuid: "54b2b48fdddf4c6ea10ae001959e2ad3" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 43611 } }
I20260806 22:52:40.691463 13915 log.cc:824] T 8ab758439aee4c089e718a92790961d5 P 54b2b48fdddf4c6ea10ae001959e2ad3: Log is configured to *not* fsync() on all Append() calls
I20260806 22:52:40.691641 13912 log.cc:824] T 8ab758439aee4c089e718a92790961d5 P 60837b341f5d4b3eb7e6a53780b4abd3: Log is configured to *not* fsync() on all Append() calls
I20260806 22:52:40.691723 13913 ts_tablet_manager.cc:1428] T 8ab758439aee4c089e718a92790961d5 P e9d32213de704457865ff3b95d803f34: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260806 22:52:40.691774 13752 heartbeater.cc:503] Master 127.28.218.254:35241 was elected leader, sending a full tablet report...
I20260806 22:52:40.692170 13915 tablet_bootstrap.cc:492] T 8ab758439aee4c089e718a92790961d5 P 54b2b48fdddf4c6ea10ae001959e2ad3: No bootstrap required, opened a new log
I20260806 22:52:40.692237 13915 ts_tablet_manager.cc:1397] T 8ab758439aee4c089e718a92790961d5 P 54b2b48fdddf4c6ea10ae001959e2ad3: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260806 22:52:40.692554 13912 tablet_bootstrap.cc:492] T 8ab758439aee4c089e718a92790961d5 P 60837b341f5d4b3eb7e6a53780b4abd3: No bootstrap required, opened a new log
I20260806 22:52:40.692613 13912 ts_tablet_manager.cc:1397] T 8ab758439aee4c089e718a92790961d5 P 60837b341f5d4b3eb7e6a53780b4abd3: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260806 22:52:40.693760 13915 raft_consensus.cc:348] T 8ab758439aee4c089e718a92790961d5 P 54b2b48fdddf4c6ea10ae001959e2ad3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60837b341f5d4b3eb7e6a53780b4abd3" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 38763 } } peers { permanent_uuid: "e9d32213de704457865ff3b95d803f34" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 46483 } } peers { permanent_uuid: "54b2b48fdddf4c6ea10ae001959e2ad3" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 43611 } }
I20260806 22:52:40.693917 13915 raft_consensus.cc:374] T 8ab758439aee4c089e718a92790961d5 P 54b2b48fdddf4c6ea10ae001959e2ad3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:52:40.693953 13915 raft_consensus.cc:729] T 8ab758439aee4c089e718a92790961d5 P 54b2b48fdddf4c6ea10ae001959e2ad3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54b2b48fdddf4c6ea10ae001959e2ad3, State: Initialized, Role: FOLLOWER
I20260806 22:52:40.694078 13915 consensus_queue.cc:260] T 8ab758439aee4c089e718a92790961d5 P 54b2b48fdddf4c6ea10ae001959e2ad3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60837b341f5d4b3eb7e6a53780b4abd3" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 38763 } } peers { permanent_uuid: "e9d32213de704457865ff3b95d803f34" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 46483 } } peers { permanent_uuid: "54b2b48fdddf4c6ea10ae001959e2ad3" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 43611 } }
I20260806 22:52:40.694090 13912 raft_consensus.cc:348] T 8ab758439aee4c089e718a92790961d5 P 60837b341f5d4b3eb7e6a53780b4abd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60837b341f5d4b3eb7e6a53780b4abd3" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 38763 } } peers { permanent_uuid: "e9d32213de704457865ff3b95d803f34" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 46483 } } peers { permanent_uuid: "54b2b48fdddf4c6ea10ae001959e2ad3" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 43611 } }
I20260806 22:52:40.694267 13912 raft_consensus.cc:374] T 8ab758439aee4c089e718a92790961d5 P 60837b341f5d4b3eb7e6a53780b4abd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:52:40.694317 13912 raft_consensus.cc:729] T 8ab758439aee4c089e718a92790961d5 P 60837b341f5d4b3eb7e6a53780b4abd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60837b341f5d4b3eb7e6a53780b4abd3, State: Initialized, Role: FOLLOWER
I20260806 22:52:40.694342 13915 ts_tablet_manager.cc:1428] T 8ab758439aee4c089e718a92790961d5 P 54b2b48fdddf4c6ea10ae001959e2ad3: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260806 22:52:40.694445 13610 heartbeater.cc:503] Master 127.28.218.254:35241 was elected leader, sending a full tablet report...
I20260806 22:52:40.694469 13912 consensus_queue.cc:260] T 8ab758439aee4c089e718a92790961d5 P 60837b341f5d4b3eb7e6a53780b4abd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60837b341f5d4b3eb7e6a53780b4abd3" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 38763 } } peers { permanent_uuid: "e9d32213de704457865ff3b95d803f34" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 46483 } } peers { permanent_uuid: "54b2b48fdddf4c6ea10ae001959e2ad3" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 43611 } }
I20260806 22:52:40.694748 13912 ts_tablet_manager.cc:1428] T 8ab758439aee4c089e718a92790961d5 P 60837b341f5d4b3eb7e6a53780b4abd3: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260806 22:52:40.694778 13893 heartbeater.cc:503] Master 127.28.218.254:35241 was elected leader, sending a full tablet report...
W20260806 22:52:40.727106 13753 tablet.cc:2369] T 8ab758439aee4c089e718a92790961d5 P e9d32213de704457865ff3b95d803f34: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:52:40.803303 13921 raft_consensus.cc:482] T 8ab758439aee4c089e718a92790961d5 P 60837b341f5d4b3eb7e6a53780b4abd3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:52:40.803467 13921 raft_consensus.cc:504] T 8ab758439aee4c089e718a92790961d5 P 60837b341f5d4b3eb7e6a53780b4abd3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60837b341f5d4b3eb7e6a53780b4abd3" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 38763 } } peers { permanent_uuid: "e9d32213de704457865ff3b95d803f34" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 46483 } } peers { permanent_uuid: "54b2b48fdddf4c6ea10ae001959e2ad3" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 43611 } }
I20260806 22:52:40.803917 13921 leader_election.cc:288] T 8ab758439aee4c089e718a92790961d5 P 60837b341f5d4b3eb7e6a53780b4abd3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9d32213de704457865ff3b95d803f34 (127.28.218.194:46483), 54b2b48fdddf4c6ea10ae001959e2ad3 (127.28.218.193:43611)
I20260806 22:52:40.807046 13910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:40.804295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.195:39529 (local address 127.28.218.193:43611)
0806 22:52:40.804364 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0806 22:52:40.804366 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:52:40.804489 (+ 123us) server_negotiation.cc:407] Connection header received
0806 22:52:40.804536 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:52:40.804537 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:52:40.804548 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:52:40.804607 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:52:40.804726 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:40.805184 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:40.806310 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:40.806692 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:40.806746 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:52:40.806773 (+ 27us) server_negotiation.cc:1035] Waiting for connection context
0806 22:52:40.806933 (+ 160us) server_negotiation.cc:299] Negotiation successful
0806 22:52:40.806979 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 22:52:40.807102 13908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:40.804173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.195:54521 (local address 127.28.218.194:46483)
0806 22:52:40.804218 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 22:52:40.804222 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:52:40.804236 (+ 14us) server_negotiation.cc:407] Connection header received
0806 22:52:40.804271 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:52:40.804272 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:52:40.804284 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:52:40.804379 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:52:40.804576 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:40.805235 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:40.806301 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:40.806698 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:40.806786 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:52:40.806795 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 22:52:40.806983 (+ 188us) server_negotiation.cc:299] Negotiation successful
0806 22:52:40.807024 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 22:52:40.807054 13922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:40.804211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.193:43611 (local address 127.28.218.195:39529)
0806 22:52:40.804423 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0806 22:52:40.804433 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:52:40.804491 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:52:40.804608 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:52:40.804609 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:52:40.804610 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:52:40.804672 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:52:40.804673 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:52:40.805254 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:52:40.805255 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:52:40.806212 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:52:40.806215 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:52:40.806776 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:52:40.806778 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:52:40.806856 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:52:40.806860 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:52:40.806912 (+ 52us) client_negotiation.cc:240] Negotiation successful
0806 22:52:40.806965 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":194,"mutex_wait_us":33,"thread_start_us":140,"threads_started":1}
I20260806 22:52:40.807067 13895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:40.804094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.194:46483 (local address 127.28.218.195:54521)
0806 22:52:40.804153 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0806 22:52:40.804166 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:52:40.804210 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:52:40.804402 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:52:40.804403 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:52:40.804404 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:52:40.804494 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:52:40.804497 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:52:40.805325 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:52:40.805326 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:52:40.806211 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:52:40.806213 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:52:40.806801 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:52:40.806802 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:52:40.806879 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:52:40.806881 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 22:52:40.806969 (+ 88us) client_negotiation.cc:240] Negotiation successful
0806 22:52:40.807000 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36,"mutex_wait_us":56}
I20260806 22:52:40.807760 13560 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ab758439aee4c089e718a92790961d5" candidate_uuid: "60837b341f5d4b3eb7e6a53780b4abd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "54b2b48fdddf4c6ea10ae001959e2ad3" is_pre_election: true
I20260806 22:52:40.807873 13703 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ab758439aee4c089e718a92790961d5" candidate_uuid: "60837b341f5d4b3eb7e6a53780b4abd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9d32213de704457865ff3b95d803f34" is_pre_election: true
I20260806 22:52:40.807940 13560 raft_consensus.cc:2454] T 8ab758439aee4c089e718a92790961d5 P 54b2b48fdddf4c6ea10ae001959e2ad3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60837b341f5d4b3eb7e6a53780b4abd3 in term 0.
I20260806 22:52:40.808008 13703 raft_consensus.cc:2454] T 8ab758439aee4c089e718a92790961d5 P e9d32213de704457865ff3b95d803f34 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60837b341f5d4b3eb7e6a53780b4abd3 in term 0.
I20260806 22:52:40.808168 13776 leader_election.cc:302] T 8ab758439aee4c089e718a92790961d5 P 60837b341f5d4b3eb7e6a53780b4abd3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54b2b48fdddf4c6ea10ae001959e2ad3, 60837b341f5d4b3eb7e6a53780b4abd3; no voters:
I20260806 22:52:40.808293 13921 raft_consensus.cc:2781] T 8ab758439aee4c089e718a92790961d5 P 60837b341f5d4b3eb7e6a53780b4abd3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:52:40.808350 13921 raft_consensus.cc:482] T 8ab758439aee4c089e718a92790961d5 P 60837b341f5d4b3eb7e6a53780b4abd3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:52:40.808380 13921 raft_consensus.cc:3037] T 8ab758439aee4c089e718a92790961d5 P 60837b341f5d4b3eb7e6a53780b4abd3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:52:40.809056 13921 raft_consensus.cc:504] T 8ab758439aee4c089e718a92790961d5 P 60837b341f5d4b3eb7e6a53780b4abd3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60837b341f5d4b3eb7e6a53780b4abd3" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 38763 } } peers { permanent_uuid: "e9d32213de704457865ff3b95d803f34" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 46483 } } peers { permanent_uuid: "54b2b48fdddf4c6ea10ae001959e2ad3" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 43611 } }
I20260806 22:52:40.809178 13921 leader_election.cc:288] T 8ab758439aee4c089e718a92790961d5 P 60837b341f5d4b3eb7e6a53780b4abd3 [CANDIDATE]: Term 1 election: Requested vote from peers e9d32213de704457865ff3b95d803f34 (127.28.218.194:46483), 54b2b48fdddf4c6ea10ae001959e2ad3 (127.28.218.193:43611)
I20260806 22:52:40.809327 13703 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ab758439aee4c089e718a92790961d5" candidate_uuid: "60837b341f5d4b3eb7e6a53780b4abd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9d32213de704457865ff3b95d803f34"
I20260806 22:52:40.809330 13560 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ab758439aee4c089e718a92790961d5" candidate_uuid: "60837b341f5d4b3eb7e6a53780b4abd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "54b2b48fdddf4c6ea10ae001959e2ad3"
I20260806 22:52:40.809406 13560 raft_consensus.cc:3037] T 8ab758439aee4c089e718a92790961d5 P 54b2b48fdddf4c6ea10ae001959e2ad3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:52:40.809406 13703 raft_consensus.cc:3037] T 8ab758439aee4c089e718a92790961d5 P e9d32213de704457865ff3b95d803f34 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:52:40.810250 13560 raft_consensus.cc:2454] T 8ab758439aee4c089e718a92790961d5 P 54b2b48fdddf4c6ea10ae001959e2ad3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60837b341f5d4b3eb7e6a53780b4abd3 in term 1.
I20260806 22:52:40.810250 13703 raft_consensus.cc:2454] T 8ab758439aee4c089e718a92790961d5 P e9d32213de704457865ff3b95d803f34 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60837b341f5d4b3eb7e6a53780b4abd3 in term 1.
I20260806 22:52:40.810422 13776 leader_election.cc:302] T 8ab758439aee4c089e718a92790961d5 P 60837b341f5d4b3eb7e6a53780b4abd3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54b2b48fdddf4c6ea10ae001959e2ad3, 60837b341f5d4b3eb7e6a53780b4abd3; no voters:
I20260806 22:52:40.810526 13921 raft_consensus.cc:2781] T 8ab758439aee4c089e718a92790961d5 P 60837b341f5d4b3eb7e6a53780b4abd3 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:52:40.810689 13921 raft_consensus.cc:686] T 8ab758439aee4c089e718a92790961d5 P 60837b341f5d4b3eb7e6a53780b4abd3 [term 1 LEADER]: Becoming Leader. State: Replica: 60837b341f5d4b3eb7e6a53780b4abd3, State: Running, Role: LEADER
I20260806 22:52:40.810786 13921 consensus_queue.cc:237] T 8ab758439aee4c089e718a92790961d5 P 60837b341f5d4b3eb7e6a53780b4abd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60837b341f5d4b3eb7e6a53780b4abd3" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 38763 } } peers { permanent_uuid: "e9d32213de704457865ff3b95d803f34" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 46483 } } peers { permanent_uuid: "54b2b48fdddf4c6ea10ae001959e2ad3" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 43611 } }
I20260806 22:52:40.811657 13410 catalog_manager.cc:5495] T 8ab758439aee4c089e718a92790961d5 P 60837b341f5d4b3eb7e6a53780b4abd3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 60837b341f5d4b3eb7e6a53780b4abd3 (127.28.218.195). New cstate: current_term: 1 leader_uuid: "60837b341f5d4b3eb7e6a53780b4abd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60837b341f5d4b3eb7e6a53780b4abd3" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 38763 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9d32213de704457865ff3b95d803f34" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 46483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54b2b48fdddf4c6ea10ae001959e2ad3" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 43611 } health_report { overall_health: UNKNOWN } } }
W20260806 22:52:40.821841 13611 tablet.cc:2369] T 8ab758439aee4c089e718a92790961d5 P 54b2b48fdddf4c6ea10ae001959e2ad3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:52:40.827176 13911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:40.825092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39858 (local address 127.28.218.195:38763)
0806 22:52:40.825163 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0806 22:52:40.825165 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:52:40.825172 (+ 7us) server_negotiation.cc:407] Connection header received
0806 22:52:40.825199 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:52:40.825200 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:52:40.825214 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:52:40.825270 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:52:40.825401 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:40.825818 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:40.826498 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:52:40.826685 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:52:40.826733 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:52:40.826867 (+ 134us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:52:40.826961 (+ 94us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:52:40.827019 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 22:52:40.827057 (+ 38us) server_negotiation.cc:299] Negotiation successful
0806 22:52:40.827073 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 22:52:40.828904 13560 raft_consensus.cc:1261] T 8ab758439aee4c089e718a92790961d5 P 54b2b48fdddf4c6ea10ae001959e2ad3 [term 1 FOLLOWER]: Refusing update from remote peer 60837b341f5d4b3eb7e6a53780b4abd3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:52:40.828965 13703 raft_consensus.cc:1261] T 8ab758439aee4c089e718a92790961d5 P e9d32213de704457865ff3b95d803f34 [term 1 FOLLOWER]: Refusing update from remote peer 60837b341f5d4b3eb7e6a53780b4abd3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:52:40.829313 13921 consensus_queue.cc:1035] T 8ab758439aee4c089e718a92790961d5 P 60837b341f5d4b3eb7e6a53780b4abd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54b2b48fdddf4c6ea10ae001959e2ad3" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 43611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:52:40.829440 13923 consensus_queue.cc:1035] T 8ab758439aee4c089e718a92790961d5 P 60837b341f5d4b3eb7e6a53780b4abd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9d32213de704457865ff3b95d803f34" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 46483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:52:40.832012 13927 mvcc.cc:205] Tried to move back new op lower bound from 7315688492352614400 to 7315688492281729024. Current Snapshot: MvccSnapshot[applied={T|T < 7315688492352614400}]
I20260806 22:52:40.832695 13928 mvcc.cc:205] Tried to move back new op lower bound from 7315688492352614400 to 7315688492281729024. Current Snapshot: MvccSnapshot[applied={T|T < 7315688492352614400}]
I20260806 22:52:40.833027 13929 mvcc.cc:205] Tried to move back new op lower bound from 7315688492352614400 to 7315688492281729024. Current Snapshot: MvccSnapshot[applied={T|T < 7315688492352614400}]
I20260806 22:52:40.834110 13410 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56108:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:52:40.834197 13410 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56108:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:52:40.834968 13410 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 97cb235f35f847ba97cf43d0dd94c0b5: Sending DeleteTablet for 3 replicas of tablet 8ab758439aee4c089e718a92790961d5
I20260806 22:52:40.835323 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 13475
I20260806 22:52:40.835309 13823 tablet_service.cc:1514] Processing DeleteTablet for tablet 8ab758439aee4c089e718a92790961d5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:52:40 UTC) from {username='kudu', principal='kudu/127.28.218.254@KRBTEST.COM'} at 127.0.0.1:39848
I20260806 22:52:40.835377 13682 tablet_service.cc:1514] Processing DeleteTablet for tablet 8ab758439aee4c089e718a92790961d5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:52:40 UTC) from {username='kudu', principal='kudu/127.28.218.254@KRBTEST.COM'} at 127.0.0.1:37462
I20260806 22:52:40.837596 13939 tablet_replica.cc:330] T 8ab758439aee4c089e718a92790961d5 P e9d32213de704457865ff3b95d803f34: stopping tablet replica
I20260806 22:52:40.837726 13939 raft_consensus.cc:2229] T 8ab758439aee4c089e718a92790961d5 P e9d32213de704457865ff3b95d803f34 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:52:40.837846 13939 raft_consensus.cc:2258] T 8ab758439aee4c089e718a92790961d5 P e9d32213de704457865ff3b95d803f34 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:52:40.838030 13938 tablet_replica.cc:330] T 8ab758439aee4c089e718a92790961d5 P 60837b341f5d4b3eb7e6a53780b4abd3: stopping tablet replica
I20260806 22:52:40.838120 13938 raft_consensus.cc:2229] T 8ab758439aee4c089e718a92790961d5 P 60837b341f5d4b3eb7e6a53780b4abd3 [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:52:40.838213 13938 raft_consensus.cc:2258] T 8ab758439aee4c089e718a92790961d5 P 60837b341f5d4b3eb7e6a53780b4abd3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:52:40.838424 13939 ts_tablet_manager.cc:1905] T 8ab758439aee4c089e718a92790961d5 P e9d32213de704457865ff3b95d803f34: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:52:40.838616 13938 ts_tablet_manager.cc:1905] T 8ab758439aee4c089e718a92790961d5 P 60837b341f5d4b3eb7e6a53780b4abd3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:52:40.840497 13939 ts_tablet_manager.cc:1918] T 8ab758439aee4c089e718a92790961d5 P e9d32213de704457865ff3b95d803f34: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:52:40.840631 13939 log.cc:1196] T 8ab758439aee4c089e718a92790961d5 P e9d32213de704457865ff3b95d803f34: Deleting WAL directory at /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-1/wal/wals/8ab758439aee4c089e718a92790961d5
I20260806 22:52:40.840658 13938 ts_tablet_manager.cc:1918] T 8ab758439aee4c089e718a92790961d5 P 60837b341f5d4b3eb7e6a53780b4abd3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:52:40.840749 13938 log.cc:1196] T 8ab758439aee4c089e718a92790961d5 P 60837b341f5d4b3eb7e6a53780b4abd3: Deleting WAL directory at /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-2/wal/wals/8ab758439aee4c089e718a92790961d5
I20260806 22:52:40.841051 13938 ts_tablet_manager.cc:1939] T 8ab758439aee4c089e718a92790961d5 P 60837b341f5d4b3eb7e6a53780b4abd3: Deleting consensus metadata
I20260806 22:52:40.841137 13939 ts_tablet_manager.cc:1939] T 8ab758439aee4c089e718a92790961d5 P e9d32213de704457865ff3b95d803f34: Deleting consensus metadata
I20260806 22:52:40.841599 13397 catalog_manager.cc:4841] TS e9d32213de704457865ff3b95d803f34 (127.28.218.194:46483): tablet 8ab758439aee4c089e718a92790961d5 (table test-table [id=b8c70b25d766460ebac7dc881afc300a]) successfully deleted
I20260806 22:52:40.841807 13396 catalog_manager.cc:4841] TS 60837b341f5d4b3eb7e6a53780b4abd3 (127.28.218.195:38763): tablet 8ab758439aee4c089e718a92790961d5 (table test-table [id=b8c70b25d766460ebac7dc881afc300a]) successfully deleted
W20260806 22:52:40.843427 13397 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.28.218.193:43611 (error 108)
I20260806 22:52:40.843722 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 13615
I20260806 22:52:40.843943 13909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:52:40.843811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.193:43611 (local address 127.0.0.1:45732)
0806 22:52:40.843847 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0806 22:52:40.843907 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.193:43611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
W20260806 22:52:40.844492 13397 catalog_manager.cc:4584] TS 54b2b48fdddf4c6ea10ae001959e2ad3 (127.28.218.193:43611): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8ab758439aee4c089e718a92790961d5: Network error: Client connection negotiation failed: client connection to 127.28.218.193:43611: connect: Connection refused (error 111)
I20260806 22:52:40.851409 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 13757
I20260806 22:52:40.858479 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 13376
I20260806 22:53:00.864774 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.218.254:35241
--webserver_interface=127.28.218.254
--webserver_port=35267
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab
--principal=kudu/127.28.218.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.218.212:40995
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.218.254:35241
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:53:00.946712 13941 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:53:00.946911 13941 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:53:00.946940 13941 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:53:00.948138 13941 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 22:53:00.948180 13941 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 22:53:00.948194 13941 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 22:53:00.948206 13941 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:53:00.948253 13941 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:53:00.948283 13941 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:53:00.948299 13941 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:53:00.948323 13941 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:53:00.949793 13941 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:40995
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.28.218.254:35241
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab
--principal=kudu/127.28.218.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.218.254:35241
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.218.254
--webserver_port=35267
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13941
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:53:00.950034 13941 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:53:00.950286 13941 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:53:00.959326 13949 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:53:00.959332 13947 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:53:00.959561 13951 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:53:00.959666 13941 server_base.cc:1034] running on GCE node
I20260806 22:53:00.959889 13941 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:53:00.960162 13941 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:53:00.961333 13941 hybrid_clock.cc:648] HybridClock initialized: now 1786056780961301 us; error 41 us; skew 500 ppm
Aug 06 22:53:00 dist-test-slave-1xrh krb5kdc[13362](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056780, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:53:00.968583 13941 init.cc:377] Logged in from keytab as kudu/127.28.218.254@KRBTEST.COM (short username kudu)
I20260806 22:53:00.969098 13941 webserver.cc:460] Webserver started at http://127.28.218.254:35267/ using document root <none> and password file <none>
I20260806 22:53:00.969364 13941 fs_manager.cc:359] Metadata directory not provided
I20260806 22:53:00.969455 13941 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:53:00.970832 13941 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:53:00.971573 13958 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:53:00.971761 13941 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 22:53:00.971869 13941 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/master-0/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/master-0/wal
uuid: "97cb235f35f847ba97cf43d0dd94c0b5"
format_stamp: "Formatted at 2026-08-06 22:52:40 on dist-test-slave-1xrh"
server_key: "0b6344b64d49e81ccfbc16cbdecc3682"
server_key_iv: "8dff30e4a2f249b76ea801a7289e364f"
server_key_version: "encryptionkey@0"
I20260806 22:53:00.972302 13941 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:53:00.984174 13941 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:53:00.986545 13941 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:53:00.986709 13941 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:53:00.991181 13941 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.254:35241
I20260806 22:53:00.991240 14012 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.254:35241 every 8 connection(s)
I20260806 22:53:00.991628 13941 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
I20260806 22:53:00.993095 14013 sys_catalog.cc:263] Verifying existing consensus state
I20260806 22:53:00.993777 14013 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97cb235f35f847ba97cf43d0dd94c0b5: Bootstrap starting.
I20260806 22:53:01.000757 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 13941
I20260806 22:53:01.001084 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.218.193:43611
--local_ip_for_outbound_sockets=127.28.218.193
--tserver_master_addrs=127.28.218.254:35241
--webserver_port=35237
--webserver_interface=127.28.218.193
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.193.keytab
--principal=kudu/127.28.218.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.218.212:40995
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:53:01.002559 14013 log.cc:824] T 00000000000000000000000000000000 P 97cb235f35f847ba97cf43d0dd94c0b5: Log is configured to *not* fsync() on all Append() calls
I20260806 22:53:01.004619 14013 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97cb235f35f847ba97cf43d0dd94c0b5: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 22:53:01.004941 14013 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97cb235f35f847ba97cf43d0dd94c0b5: Bootstrap complete.
I20260806 22:53:01.007315 14013 raft_consensus.cc:348] T 00000000000000000000000000000000 P 97cb235f35f847ba97cf43d0dd94c0b5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97cb235f35f847ba97cf43d0dd94c0b5" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 35241 } }
I20260806 22:53:01.007577 14013 raft_consensus.cc:729] T 00000000000000000000000000000000 P 97cb235f35f847ba97cf43d0dd94c0b5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97cb235f35f847ba97cf43d0dd94c0b5, State: Initialized, Role: FOLLOWER
I20260806 22:53:01.007764 14013 consensus_queue.cc:260] T 00000000000000000000000000000000 P 97cb235f35f847ba97cf43d0dd94c0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97cb235f35f847ba97cf43d0dd94c0b5" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 35241 } }
I20260806 22:53:01.007843 14013 raft_consensus.cc:388] T 00000000000000000000000000000000 P 97cb235f35f847ba97cf43d0dd94c0b5 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:53:01.007903 14013 raft_consensus.cc:482] T 00000000000000000000000000000000 P 97cb235f35f847ba97cf43d0dd94c0b5 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:53:01.007958 14013 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 97cb235f35f847ba97cf43d0dd94c0b5 [term 1 FOLLOWER]: Advancing to term 2
I20260806 22:53:01.008970 14013 raft_consensus.cc:504] T 00000000000000000000000000000000 P 97cb235f35f847ba97cf43d0dd94c0b5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97cb235f35f847ba97cf43d0dd94c0b5" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 35241 } }
I20260806 22:53:01.009096 14013 leader_election.cc:302] T 00000000000000000000000000000000 P 97cb235f35f847ba97cf43d0dd94c0b5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 97cb235f35f847ba97cf43d0dd94c0b5; no voters:
I20260806 22:53:01.009312 14013 leader_election.cc:288] T 00000000000000000000000000000000 P 97cb235f35f847ba97cf43d0dd94c0b5 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 22:53:01.009356 14017 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 97cb235f35f847ba97cf43d0dd94c0b5 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 22:53:01.009473 14017 raft_consensus.cc:686] T 00000000000000000000000000000000 P 97cb235f35f847ba97cf43d0dd94c0b5 [term 2 LEADER]: Becoming Leader. State: Replica: 97cb235f35f847ba97cf43d0dd94c0b5, State: Running, Role: LEADER
I20260806 22:53:01.009562 14017 consensus_queue.cc:237] T 00000000000000000000000000000000 P 97cb235f35f847ba97cf43d0dd94c0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97cb235f35f847ba97cf43d0dd94c0b5" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 35241 } }
I20260806 22:53:01.009596 14013 sys_catalog.cc:564] T 00000000000000000000000000000000 P 97cb235f35f847ba97cf43d0dd94c0b5 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:53:01.009816 14017 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97cb235f35f847ba97cf43d0dd94c0b5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 97cb235f35f847ba97cf43d0dd94c0b5. Latest consensus state: current_term: 2 leader_uuid: "97cb235f35f847ba97cf43d0dd94c0b5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97cb235f35f847ba97cf43d0dd94c0b5" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 35241 } } }
I20260806 22:53:01.009907 14017 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97cb235f35f847ba97cf43d0dd94c0b5 [sys.catalog]: This master's current role is: LEADER
I20260806 22:53:01.009824 14018 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97cb235f35f847ba97cf43d0dd94c0b5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "97cb235f35f847ba97cf43d0dd94c0b5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97cb235f35f847ba97cf43d0dd94c0b5" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 35241 } } }
I20260806 22:53:01.010108 14018 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97cb235f35f847ba97cf43d0dd94c0b5 [sys.catalog]: This master's current role is: LEADER
I20260806 22:53:01.010115 14021 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:53:01.011171 14021 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:53:01.011294 14021 catalog_manager.cc:1259] Loaded cluster ID: 3c7f71f6744b4458a6f069c02919cdd6
I20260806 22:53:01.011334 14021 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:53:01.012676 14021 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:53:01.012861 14021 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 97cb235f35f847ba97cf43d0dd94c0b5: Loaded TSK: 0
I20260806 22:53:01.029331 14021 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 97cb235f35f847ba97cf43d0dd94c0b5: Generated new TSK 1
I20260806 22:53:01.029421 14021 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 22:53:01.089552 14015 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:53:01.089736 14015 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:53:01.089758 14015 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:53:01.090857 14015 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:53:01.090896 14015 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:53:01.090920 14015 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.218.193
I20260806 22:53:01.092370 14015 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:40995
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.193.keytab
--principal=kudu/127.28.218.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.218.193:43611
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.218.193
--webserver_port=35237
--tserver_master_addrs=127.28.218.254:35241
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14015
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.218.193
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:53:01.092598 14015 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:53:01.092840 14015 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:53:01.098850 14042 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:53:01.098969 14046 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:53:01.098901 14043 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:53:01.099078 14015 server_base.cc:1034] running on GCE node
I20260806 22:53:01.099300 14015 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:53:01.099536 14015 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:53:01.100708 14015 hybrid_clock.cc:648] HybridClock initialized: now 1786056781100692 us; error 32 us; skew 500 ppm
Aug 06 22:53:01 dist-test-slave-1xrh krb5kdc[13362](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056781, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:53:01.103365 14015 init.cc:377] Logged in from keytab as kudu/127.28.218.193@KRBTEST.COM (short username kudu)
I20260806 22:53:01.103824 14015 webserver.cc:460] Webserver started at http://127.28.218.193:35237/ using document root <none> and password file <none>
I20260806 22:53:01.104053 14015 fs_manager.cc:359] Metadata directory not provided
I20260806 22:53:01.104137 14015 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:53:01.105556 14015 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:53:01.106282 14053 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:53:01.106477 14015 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:53:01.106544 14015 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-0/wal
uuid: "54b2b48fdddf4c6ea10ae001959e2ad3"
format_stamp: "Formatted at 2026-08-06 22:52:40 on dist-test-slave-1xrh"
server_key: "3a5eced6bd4526f6244d49e99c177928"
server_key_iv: "6dc9cd2527d76c346819058f2dfbdd57"
server_key_version: "encryptionkey@0"
I20260806 22:53:01.106877 14015 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:53:01.120471 14015 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:53:01.122651 14015 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:53:01.122792 14015 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:53:01.123021 14015 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:53:01.123478 14060 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 22:53:01.125532 14015 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 22:53:01.125593 14015 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260806 22:53:01.125649 14015 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 22:53:01.126259 14015 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 22:53:01.126312 14015 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260806 22:53:01.126391 14060 tablet_bootstrap.cc:492] T 8ab758439aee4c089e718a92790961d5 P 54b2b48fdddf4c6ea10ae001959e2ad3: Bootstrap starting.
I20260806 22:53:01.133807 14015 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.193:43611
I20260806 22:53:01.133963 14171 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.193:43611 every 8 connection(s)
I20260806 22:53:01.134276 14015 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-0/data/info.pb
I20260806 22:53:01.136847 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 14015
I20260806 22:53:01.136905 14060 log.cc:824] T 8ab758439aee4c089e718a92790961d5 P 54b2b48fdddf4c6ea10ae001959e2ad3: Log is configured to *not* fsync() on all Append() calls
I20260806 22:53:01.137162 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.218.194:46483
--local_ip_for_outbound_sockets=127.28.218.194
--tserver_master_addrs=127.28.218.254:35241
--webserver_port=42403
--webserver_interface=127.28.218.194
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.194.keytab
--principal=kudu/127.28.218.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.218.212:40995
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:53:01.139845 14060 tablet_bootstrap.cc:492] T 8ab758439aee4c089e718a92790961d5 P 54b2b48fdddf4c6ea10ae001959e2ad3: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 22:53:01.140115 14060 tablet_bootstrap.cc:492] T 8ab758439aee4c089e718a92790961d5 P 54b2b48fdddf4c6ea10ae001959e2ad3: Bootstrap complete.
I20260806 22:53:01.141011 14060 ts_tablet_manager.cc:1397] T 8ab758439aee4c089e718a92790961d5 P 54b2b48fdddf4c6ea10ae001959e2ad3: Time spent bootstrapping tablet: real 0.015s user 0.004s sys 0.009s
I20260806 22:53:01.142704 14060 raft_consensus.cc:348] T 8ab758439aee4c089e718a92790961d5 P 54b2b48fdddf4c6ea10ae001959e2ad3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60837b341f5d4b3eb7e6a53780b4abd3" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 38763 } } peers { permanent_uuid: "e9d32213de704457865ff3b95d803f34" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 46483 } } peers { permanent_uuid: "54b2b48fdddf4c6ea10ae001959e2ad3" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 43611 } }
I20260806 22:53:01.143078 14060 raft_consensus.cc:729] T 8ab758439aee4c089e718a92790961d5 P 54b2b48fdddf4c6ea10ae001959e2ad3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54b2b48fdddf4c6ea10ae001959e2ad3, State: Initialized, Role: FOLLOWER
I20260806 22:53:01.143266 14060 consensus_queue.cc:260] T 8ab758439aee4c089e718a92790961d5 P 54b2b48fdddf4c6ea10ae001959e2ad3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60837b341f5d4b3eb7e6a53780b4abd3" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 38763 } } peers { permanent_uuid: "e9d32213de704457865ff3b95d803f34" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 46483 } } peers { permanent_uuid: "54b2b48fdddf4c6ea10ae001959e2ad3" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 43611 } }
I20260806 22:53:01.143576 14060 ts_tablet_manager.cc:1428] T 8ab758439aee4c089e718a92790961d5 P 54b2b48fdddf4c6ea10ae001959e2ad3: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
Aug 06 22:53:01 dist-test-slave-1xrh krb5kdc[13362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056781, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.193@KRBTEST.COM for kudu/127.28.218.254@KRBTEST.COM
I20260806 22:53:01.147717 14174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:01.135009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:35241 (local address 127.28.218.193:56251)
0806 22:53:01.135207 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0806 22:53:01.135255 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 22:53:01.135768 (+ 513us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:53:01.138560 (+ 2792us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:53:01.138564 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:53:01.138932 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:53:01.139212 (+ 280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:53:01.139218 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:53:01.140323 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:53:01.140325 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:53:01.141157 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:53:01.141160 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:53:01.141802 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:53:01.141803 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:53:01.141940 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:01.142475 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:53:01.142491 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:53:01.144057 (+ 1566us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:53:01.146055 (+ 1998us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:53:01.146062 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:53:01.146063 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:53:01.146292 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:53:01.146579 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:53:01.146580 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:53:01.146581 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:53:01.146693 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:53:01.147073 (+ 380us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:53:01.147079 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:53:01.147240 (+ 161us) client_negotiation.cc:769] Sending connection context
0806 22:53:01.147341 (+ 101us) client_negotiation.cc:240] Negotiation successful
0806 22:53:01.147517 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":105,"threads_started":1}
I20260806 22:53:01.147717 14176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:01.135216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.193:56251 (local address 127.28.218.254:35241)
0806 22:53:01.137239 (+ 2023us) server_negotiation.cc:206] Beginning negotiation
0806 22:53:01.137248 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:53:01.137284 (+ 36us) server_negotiation.cc:407] Connection header received
0806 22:53:01.137420 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:53:01.137427 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:53:01.137480 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:53:01.138494 (+ 1014us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:53:01.139338 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:01.140224 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:01.141246 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:01.141687 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:01.141820 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:01.144171 (+ 2351us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:53:01.144200 (+ 29us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:53:01.144201 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:53:01.144221 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:53:01.145939 (+ 1718us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:53:01.146394 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:53:01.146398 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:53:01.146398 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:53:01.146492 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:53:01.146769 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:53:01.146770 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:53:01.146770 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:53:01.146994 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:53:01.147089 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 22:53:01.147414 (+ 325us) server_negotiation.cc:299] Negotiation successful
0806 22:53:01.147552 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1973,"thread_start_us":83,"threads_started":1}
I20260806 22:53:01.148362 14172 heartbeater.cc:347] Connected to a master server at 127.28.218.254:35241
I20260806 22:53:01.148453 14172 heartbeater.cc:464] Registering TS with master...
I20260806 22:53:01.148644 14172 heartbeater.cc:511] Master 127.28.218.254:35241 requested a full tablet report, sending...
I20260806 22:53:01.149338 13976 ts_manager.cc:194] Registered new tserver with Master: 54b2b48fdddf4c6ea10ae001959e2ad3 (127.28.218.193:43611)
I20260806 22:53:01.150460 13976 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.218.193@KRBTEST.COM'} at 127.28.218.193:56251
I20260806 22:53:01.151403 14172 heartbeater.cc:503] Master 127.28.218.254:35241 was elected leader, sending a full tablet report...
Aug 06 22:53:01 dist-test-slave-1xrh krb5kdc[13362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056780, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.254@KRBTEST.COM for kudu/127.28.218.193@KRBTEST.COM
I20260806 22:53:01.157961 14180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:01.149889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.193:43611 (local address 127.0.0.1:36068)
0806 22:53:01.150054 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0806 22:53:01.150099 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 22:53:01.150168 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:53:01.150403 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:53:01.150408 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:53:01.150719 (+ 311us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:53:01.150770 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:53:01.150771 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:53:01.152073 (+ 1302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:53:01.152074 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:53:01.152681 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:53:01.152683 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:53:01.153294 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:53:01.153295 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:53:01.153351 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:01.153839 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:53:01.153844 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:53:01.155030 (+ 1186us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:53:01.157009 (+ 1979us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:53:01.157015 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:53:01.157016 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:53:01.157201 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:53:01.157402 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:53:01.157403 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:53:01.157403 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:53:01.157451 (+ 48us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:53:01.157715 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:53:01.157720 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:53:01.157754 (+ 34us) client_negotiation.cc:769] Sending connection context
0806 22:53:01.157810 (+ 56us) client_negotiation.cc:240] Negotiation successful
0806 22:53:01.157888 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":80,"threads_started":1}
I20260806 22:53:01.158092 14181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:01.150110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36068 (local address 127.28.218.193:43611)
0806 22:53:01.150237 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0806 22:53:01.150241 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:53:01.150250 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:53:01.150279 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:53:01.150281 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:53:01.150292 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:53:01.150361 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:53:01.150884 (+ 523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:01.151993 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:01.152763 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:01.153215 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:01.153278 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:01.155115 (+ 1837us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:53:01.155150 (+ 35us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:53:01.155151 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:53:01.155158 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:53:01.156920 (+ 1762us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:53:01.157273 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:53:01.157275 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:53:01.157276 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:53:01.157335 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:53:01.157503 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:53:01.157504 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:53:01.157504 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:53:01.157657 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:53:01.157734 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 22:53:01.157858 (+ 124us) server_negotiation.cc:299] Negotiation successful
0806 22:53:01.157954 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":66,"threads_started":1}
I20260806 22:53:01.158401 14105 tablet_service.cc:1514] Processing DeleteTablet for tablet 8ab758439aee4c089e718a92790961d5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:52:40 UTC) from {username='kudu', principal='kudu/127.28.218.254@KRBTEST.COM'} at 127.0.0.1:36068
I20260806 22:53:01.158877 14182 tablet_replica.cc:330] stopping tablet replica
I20260806 22:53:01.158984 14182 raft_consensus.cc:2229] T 8ab758439aee4c089e718a92790961d5 P 54b2b48fdddf4c6ea10ae001959e2ad3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:53:01.159054 14182 raft_consensus.cc:2258] T 8ab758439aee4c089e718a92790961d5 P 54b2b48fdddf4c6ea10ae001959e2ad3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:53:01.159364 14182 ts_tablet_manager.cc:1905] T 8ab758439aee4c089e718a92790961d5 P 54b2b48fdddf4c6ea10ae001959e2ad3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:53:01.161404 14182 ts_tablet_manager.cc:1918] T 8ab758439aee4c089e718a92790961d5 P 54b2b48fdddf4c6ea10ae001959e2ad3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:53:01.161464 14182 log.cc:1196] T 8ab758439aee4c089e718a92790961d5 P 54b2b48fdddf4c6ea10ae001959e2ad3: Deleting WAL directory at /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-0/wal/wals/8ab758439aee4c089e718a92790961d5
I20260806 22:53:01.161691 14182 ts_tablet_manager.cc:1939] T 8ab758439aee4c089e718a92790961d5 P 54b2b48fdddf4c6ea10ae001959e2ad3: Deleting consensus metadata
I20260806 22:53:01.162153 13962 catalog_manager.cc:4841] TS 54b2b48fdddf4c6ea10ae001959e2ad3 (127.28.218.193:43611): tablet 8ab758439aee4c089e718a92790961d5 (table test-table [id=b8c70b25d766460ebac7dc881afc300a]) successfully deleted
W20260806 22:53:01.223572 14178 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:53:01.223785 14178 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:53:01.223858 14178 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:53:01.225042 14178 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:53:01.225111 14178 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:53:01.225157 14178 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.218.194
I20260806 22:53:01.226568 14178 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:40995
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.194.keytab
--principal=kudu/127.28.218.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.218.194:46483
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.218.194
--webserver_port=42403
--tserver_master_addrs=127.28.218.254:35241
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14178
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.218.194
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:53:01.226842 14178 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:53:01.227063 14178 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:53:01.231889 14191 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:53:01.231914 14187 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:53:01.231860 14188 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:53:01.232113 14178 server_base.cc:1034] running on GCE node
I20260806 22:53:01.232388 14178 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:53:01.232754 14178 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:53:01.233985 14178 hybrid_clock.cc:648] HybridClock initialized: now 1786056781233977 us; error 33 us; skew 500 ppm
Aug 06 22:53:01 dist-test-slave-1xrh krb5kdc[13362](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056781, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:53:01.236677 14178 init.cc:377] Logged in from keytab as kudu/127.28.218.194@KRBTEST.COM (short username kudu)
I20260806 22:53:01.237154 14178 webserver.cc:460] Webserver started at http://127.28.218.194:42403/ using document root <none> and password file <none>
I20260806 22:53:01.237372 14178 fs_manager.cc:359] Metadata directory not provided
I20260806 22:53:01.237473 14178 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:53:01.238839 14178 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:53:01.239548 14198 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:53:01.239717 14178 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 22:53:01.239810 14178 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-1/wal
uuid: "e9d32213de704457865ff3b95d803f34"
format_stamp: "Formatted at 2026-08-06 22:52:40 on dist-test-slave-1xrh"
server_key: "7de9e2e87fdbfc25d45f21e606f19973"
server_key_iv: "72954717981183c76dfab68b323d4e36"
server_key_version: "encryptionkey@0"
I20260806 22:53:01.240159 14178 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:53:01.258610 14178 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:53:01.260439 14178 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:53:01.260588 14178 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:53:01.260800 14178 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:53:01.261102 14178 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:53:01.261161 14178 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:53:01.261204 14178 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:53:01.261232 14178 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:53:01.267231 14178 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.194:46483
I20260806 22:53:01.267282 14316 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.194:46483 every 8 connection(s)
I20260806 22:53:01.267601 14178 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-1/data/info.pb
Aug 06 22:53:01 dist-test-slave-1xrh krb5kdc[13362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056781, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.194@KRBTEST.COM for kudu/127.28.218.254@KRBTEST.COM
I20260806 22:53:01.274748 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 14178
I20260806 22:53:01.275012 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.218.195:38763
--local_ip_for_outbound_sockets=127.28.218.195
--tserver_master_addrs=127.28.218.254:35241
--webserver_port=44067
--webserver_interface=127.28.218.195
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.195.keytab
--principal=kudu/127.28.218.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.218.212:40995
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:53:01.277030 14176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:01.268231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.194:45505 (local address 127.28.218.254:35241)
0806 22:53:01.268322 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0806 22:53:01.268326 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:53:01.268512 (+ 186us) server_negotiation.cc:407] Connection header received
0806 22:53:01.269029 (+ 517us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:53:01.269031 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:53:01.269042 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:53:01.269107 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:53:01.270030 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:01.270536 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:01.271590 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:01.271818 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:01.271906 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:01.273978 (+ 2072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:53:01.273980 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:53:01.273980 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:53:01.273988 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:53:01.275618 (+ 1630us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:53:01.276012 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:53:01.276013 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:53:01.276013 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:53:01.276076 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:53:01.276335 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:53:01.276336 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:53:01.276336 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:53:01.276454 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:53:01.276529 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 22:53:01.276887 (+ 358us) server_negotiation.cc:299] Negotiation successful
0806 22:53:01.276963 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 22:53:01.277295 14319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:01.268209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:35241 (local address 127.28.218.194:45505)
0806 22:53:01.268389 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0806 22:53:01.268424 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 22:53:01.268916 (+ 492us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:53:01.269184 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:53:01.269188 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:53:01.269744 (+ 556us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:53:01.269920 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:53:01.269924 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:53:01.270642 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:53:01.270643 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:53:01.271498 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:53:01.271500 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:53:01.271915 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:53:01.271917 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:53:01.272046 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:01.272564 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:53:01.272573 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:53:01.273881 (+ 1308us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:53:01.275703 (+ 1822us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:53:01.275710 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:53:01.275712 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:53:01.275933 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:53:01.276145 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:53:01.276145 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:53:01.276146 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:53:01.276258 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:53:01.276520 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:53:01.276527 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:53:01.276693 (+ 166us) client_negotiation.cc:769] Sending connection context
0806 22:53:01.276832 (+ 139us) client_negotiation.cc:240] Negotiation successful
0806 22:53:01.277021 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":104,"threads_started":1}
I20260806 22:53:01.277801 14317 heartbeater.cc:347] Connected to a master server at 127.28.218.254:35241
I20260806 22:53:01.277906 14317 heartbeater.cc:464] Registering TS with master...
I20260806 22:53:01.278097 14317 heartbeater.cc:511] Master 127.28.218.254:35241 requested a full tablet report, sending...
I20260806 22:53:01.278496 13976 ts_manager.cc:194] Registered new tserver with Master: e9d32213de704457865ff3b95d803f34 (127.28.218.194:46483)
I20260806 22:53:01.278878 13976 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.218.194@KRBTEST.COM'} at 127.28.218.194:45505
W20260806 22:53:01.358980 14320 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:53:01.359251 14320 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:53:01.359294 14320 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:53:01.360419 14320 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:53:01.360482 14320 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:53:01.360526 14320 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.218.195
I20260806 22:53:01.361945 14320 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:40995
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.195.keytab
--principal=kudu/127.28.218.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.218.195:38763
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.218.195
--webserver_port=44067
--tserver_master_addrs=127.28.218.254:35241
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14320
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.218.195
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:53:01.362206 14320 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:53:01.362438 14320 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:53:01.367467 14328 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:53:01.367519 14326 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:53:01.367529 14325 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:53:01.367530 14320 server_base.cc:1034] running on GCE node
I20260806 22:53:01.367877 14320 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:53:01.368166 14320 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:53:01.369325 14320 hybrid_clock.cc:648] HybridClock initialized: now 1786056781369306 us; error 38 us; skew 500 ppm
Aug 06 22:53:01 dist-test-slave-1xrh krb5kdc[13362](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056781, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:53:01.371871 14320 init.cc:377] Logged in from keytab as kudu/127.28.218.195@KRBTEST.COM (short username kudu)
I20260806 22:53:01.372306 14320 webserver.cc:460] Webserver started at http://127.28.218.195:44067/ using document root <none> and password file <none>
I20260806 22:53:01.372500 14320 fs_manager.cc:359] Metadata directory not provided
I20260806 22:53:01.372550 14320 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:53:01.373805 14320 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:53:01.374332 14335 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:53:01.374519 14320 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:53:01.374586 14320 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-2/wal
uuid: "60837b341f5d4b3eb7e6a53780b4abd3"
format_stamp: "Formatted at 2026-08-06 22:52:40 on dist-test-slave-1xrh"
server_key: "4b2e8dedab4651ff535f40daf801b17c"
server_key_iv: "dcc9a875e77dc9e6a9718036fd237485"
server_key_version: "encryptionkey@0"
I20260806 22:53:01.374980 14320 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:53:01.381999 14320 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:53:01.383736 14320 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:53:01.383915 14320 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:53:01.384120 14320 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:53:01.384429 14320 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:53:01.384464 14320 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:53:01.384483 14320 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:53:01.384537 14320 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:53:01.390259 14320 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.195:38763
I20260806 22:53:01.390302 14453 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.195:38763 every 8 connection(s)
I20260806 22:53:01.390602 14320 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-2/data/info.pb
I20260806 22:53:01.391275 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 14320
Aug 06 22:53:01 dist-test-slave-1xrh krb5kdc[13362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056781, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.195@KRBTEST.COM for kudu/127.28.218.254@KRBTEST.COM
I20260806 22:53:01.398816 14457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:01.391672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34368 (local address 127.28.218.254:35241)
0806 22:53:01.391790 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0806 22:53:01.391791 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:53:01.391931 (+ 140us) server_negotiation.cc:407] Connection header received
0806 22:53:01.391978 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:53:01.391979 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:53:01.391992 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:53:01.392064 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:53:01.392640 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:01.393424 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:01.394496 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:01.394809 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:01.394901 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:01.396213 (+ 1312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:53:01.396215 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:53:01.396215 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:53:01.396223 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:53:01.397576 (+ 1353us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:53:01.397951 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:53:01.397952 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:53:01.397953 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:53:01.398015 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:53:01.398243 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:53:01.398244 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:53:01.398245 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:53:01.398396 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:53:01.398565 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0806 22:53:01.398610 (+ 45us) server_negotiation.cc:299] Negotiation successful
0806 22:53:01.398701 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":74,"threads_started":1}
I20260806 22:53:01.401034 14176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:01.391354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.195:48273 (local address 127.28.218.254:35241)
0806 22:53:01.391413 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0806 22:53:01.391415 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:53:01.391634 (+ 219us) server_negotiation.cc:407] Connection header received
0806 22:53:01.392382 (+ 748us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:53:01.392384 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:53:01.392395 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:53:01.392446 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:53:01.393558 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:01.394268 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:01.395553 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:01.395913 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:01.395994 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:01.398284 (+ 2290us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:53:01.398285 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:53:01.398285 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:53:01.398291 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:53:01.399976 (+ 1685us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:53:01.400331 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:53:01.400331 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:53:01.400332 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:53:01.400380 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:53:01.400571 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:53:01.400572 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:53:01.400572 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:53:01.400661 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:53:01.400708 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0806 22:53:01.400916 (+ 208us) server_negotiation.cc:299] Negotiation successful
0806 22:53:01.400975 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260806 22:53:01.401178 14456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:01.391317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:35241 (local address 127.28.218.195:48273)
0806 22:53:01.391495 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0806 22:53:01.391545 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 22:53:01.392287 (+ 742us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:53:01.392510 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:53:01.392513 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:53:01.393224 (+ 711us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:53:01.393456 (+ 232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:53:01.393461 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:53:01.394354 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:53:01.394355 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:53:01.395475 (+ 1120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:53:01.395478 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:53:01.396005 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:53:01.396006 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:53:01.396105 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:01.396566 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:53:01.396576 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:53:01.398185 (+ 1609us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:53:01.400049 (+ 1864us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:53:01.400054 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:53:01.400056 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:53:01.400264 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:53:01.400421 (+ 157us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:53:01.400421 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:53:01.400421 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:53:01.400500 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:53:01.400701 (+ 201us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:53:01.400704 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:53:01.400804 (+ 100us) client_negotiation.cc:769] Sending connection context
0806 22:53:01.400879 (+ 75us) client_negotiation.cc:240] Negotiation successful
0806 22:53:01.400988 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":100,"threads_started":1}
I20260806 22:53:01.401557 14454 heartbeater.cc:347] Connected to a master server at 127.28.218.254:35241
I20260806 22:53:01.401670 14454 heartbeater.cc:464] Registering TS with master...
I20260806 22:53:01.401839 14454 heartbeater.cc:511] Master 127.28.218.254:35241 requested a full tablet report, sending...
I20260806 22:53:01.402195 13976 ts_manager.cc:194] Registered new tserver with Master: 60837b341f5d4b3eb7e6a53780b4abd3 (127.28.218.195:38763)
I20260806 22:53:01.402505 13976 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.218.195@KRBTEST.COM'} at 127.28.218.195:48273
I20260806 22:53:01.410791 29547 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:53:01.413134 14176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:01.411209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34372 (local address 127.28.218.254:35241)
0806 22:53:01.411243 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0806 22:53:01.411245 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:53:01.411306 (+ 61us) server_negotiation.cc:407] Connection header received
0806 22:53:01.411365 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:53:01.411366 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:53:01.411394 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:53:01.411458 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:53:01.411530 (+ 72us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:01.411948 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:01.412636 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:01.412797 (+ 161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:01.412844 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:01.412960 (+ 116us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:53:01.412997 (+ 37us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0806 22:53:01.413092 (+ 95us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:34372: token expired
Metrics: {"server-negotiator.queue_time_us":17}
W20260806 22:53:01.413199 14176 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:34372: token expired
W20260806 22:53:01.413350 13899 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.28.218.254:35241)
I20260806 22:53:01.418705 14176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:01.413621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34378 (local address 127.28.218.254:35241)
0806 22:53:01.413653 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0806 22:53:01.413655 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:53:01.413682 (+ 27us) server_negotiation.cc:407] Connection header received
0806 22:53:01.413724 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:53:01.413725 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:53:01.413731 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:53:01.413779 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:53:01.414197 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:01.414600 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:01.415297 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:01.415455 (+ 158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:01.415511 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:01.416549 (+ 1038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:53:01.416550 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:53:01.416550 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:53:01.416555 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:53:01.417666 (+ 1111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:53:01.418169 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:53:01.418170 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:53:01.418170 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:53:01.418216 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:53:01.418361 (+ 145us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:53:01.418361 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:53:01.418361 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:53:01.418446 (+ 85us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:53:01.418525 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 22:53:01.418602 (+ 77us) server_negotiation.cc:299] Negotiation successful
0806 22:53:01.418653 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260806 22:53:01.419962 13976 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34378:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:53:01.421795 13976 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:53:01.424031 14105 tablet_service.cc:1467] Processing CreateTablet for tablet 38b53c157d264a9d84725854c91d8697 (DEFAULT_TABLE table=test-table [id=20364c58117a4784b3948a88b2707192]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:53:01.424240 14105 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 38b53c157d264a9d84725854c91d8697. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:53:01.425582 14060 tablet_bootstrap.cc:492] T 38b53c157d264a9d84725854c91d8697 P 54b2b48fdddf4c6ea10ae001959e2ad3: Bootstrap starting.
I20260806 22:53:01.425900 14060 tablet_bootstrap.cc:654] T 38b53c157d264a9d84725854c91d8697 P 54b2b48fdddf4c6ea10ae001959e2ad3: Neither blocks nor log segments found. Creating new log.
I20260806 22:53:01.426709 14060 tablet_bootstrap.cc:492] T 38b53c157d264a9d84725854c91d8697 P 54b2b48fdddf4c6ea10ae001959e2ad3: No bootstrap required, opened a new log
I20260806 22:53:01.426784 14060 ts_tablet_manager.cc:1397] T 38b53c157d264a9d84725854c91d8697 P 54b2b48fdddf4c6ea10ae001959e2ad3: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 22:53:01.426944 14060 raft_consensus.cc:348] T 38b53c157d264a9d84725854c91d8697 P 54b2b48fdddf4c6ea10ae001959e2ad3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60837b341f5d4b3eb7e6a53780b4abd3" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 38763 } } peers { permanent_uuid: "e9d32213de704457865ff3b95d803f34" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 46483 } } peers { permanent_uuid: "54b2b48fdddf4c6ea10ae001959e2ad3" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 43611 } }
I20260806 22:53:01.427019 14060 raft_consensus.cc:374] T 38b53c157d264a9d84725854c91d8697 P 54b2b48fdddf4c6ea10ae001959e2ad3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:53:01.427073 14060 raft_consensus.cc:729] T 38b53c157d264a9d84725854c91d8697 P 54b2b48fdddf4c6ea10ae001959e2ad3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54b2b48fdddf4c6ea10ae001959e2ad3, State: Initialized, Role: FOLLOWER
I20260806 22:53:01.427117 14060 consensus_queue.cc:260] T 38b53c157d264a9d84725854c91d8697 P 54b2b48fdddf4c6ea10ae001959e2ad3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60837b341f5d4b3eb7e6a53780b4abd3" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 38763 } } peers { permanent_uuid: "e9d32213de704457865ff3b95d803f34" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 46483 } } peers { permanent_uuid: "54b2b48fdddf4c6ea10ae001959e2ad3" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 43611 } }
I20260806 22:53:01.427232 14060 ts_tablet_manager.cc:1428] T 38b53c157d264a9d84725854c91d8697 P 54b2b48fdddf4c6ea10ae001959e2ad3: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260806 22:53:01.427489 14180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:01.423539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.195:38763 (local address 127.0.0.1:38828)
0806 22:53:01.423588 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0806 22:53:01.423600 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:53:01.423640 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:53:01.424001 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:53:01.424002 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:53:01.424004 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:53:01.424046 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:53:01.424047 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:53:01.425277 (+ 1230us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:53:01.425278 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:53:01.426382 (+ 1104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:53:01.426385 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:53:01.427267 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:53:01.427268 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:53:01.427330 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:01.427350 (+ 20us) client_negotiation.cc:769] Sending connection context
0806 22:53:01.427380 (+ 30us) client_negotiation.cc:240] Negotiation successful
0806 22:53:01.427418 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260806 22:53:01.427785 14462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:01.423631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38828 (local address 127.28.218.195:38763)
0806 22:53:01.423779 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0806 22:53:01.423783 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:53:01.423793 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:53:01.423844 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:53:01.423846 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:53:01.423896 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:53:01.423982 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:53:01.424134 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:01.425176 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:01.426494 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:01.427183 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:01.427592 (+ 409us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:01.427650 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 22:53:01.427675 (+ 25us) server_negotiation.cc:299] Negotiation successful
0806 22:53:01.427709 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":90,"threads_started":1}
I20260806 22:53:01.428205 14461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:01.423597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.194:46483 (local address 127.0.0.1:51004)
0806 22:53:01.423764 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0806 22:53:01.423775 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:53:01.423836 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:53:01.424366 (+ 530us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:53:01.424368 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:53:01.424370 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:53:01.424414 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:53:01.424416 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:53:01.425844 (+ 1428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:53:01.425845 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:53:01.427013 (+ 1168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:53:01.427016 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:53:01.427964 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:53:01.427965 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:53:01.428015 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:01.428018 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:53:01.428108 (+ 90us) client_negotiation.cc:240] Negotiation successful
0806 22:53:01.428144 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":115,"threads_started":1}
I20260806 22:53:01.428242 14463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:01.423936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51004 (local address 127.28.218.194:46483)
0806 22:53:01.424145 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0806 22:53:01.424150 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:53:01.424160 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:53:01.424199 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:53:01.424201 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:53:01.424216 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:53:01.424301 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:53:01.424563 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:01.425760 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:01.427103 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:01.427815 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:01.427883 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:01.427934 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 22:53:01.428123 (+ 189us) server_negotiation.cc:299] Negotiation successful
0806 22:53:01.428163 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":123,"threads_started":1}
I20260806 22:53:01.428475 14386 tablet_service.cc:1467] Processing CreateTablet for tablet 38b53c157d264a9d84725854c91d8697 (DEFAULT_TABLE table=test-table [id=20364c58117a4784b3948a88b2707192]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:53:01.428812 14386 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 38b53c157d264a9d84725854c91d8697. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:53:01.428893 14249 tablet_service.cc:1467] Processing CreateTablet for tablet 38b53c157d264a9d84725854c91d8697 (DEFAULT_TABLE table=test-table [id=20364c58117a4784b3948a88b2707192]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:53:01.429229 14249 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 38b53c157d264a9d84725854c91d8697. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:53:01.431326 14465 tablet_bootstrap.cc:492] T 38b53c157d264a9d84725854c91d8697 P 60837b341f5d4b3eb7e6a53780b4abd3: Bootstrap starting.
I20260806 22:53:01.431867 14466 tablet_bootstrap.cc:492] T 38b53c157d264a9d84725854c91d8697 P e9d32213de704457865ff3b95d803f34: Bootstrap starting.
I20260806 22:53:01.432046 14465 tablet_bootstrap.cc:654] T 38b53c157d264a9d84725854c91d8697 P 60837b341f5d4b3eb7e6a53780b4abd3: Neither blocks nor log segments found. Creating new log.
I20260806 22:53:01.432427 14466 tablet_bootstrap.cc:654] T 38b53c157d264a9d84725854c91d8697 P e9d32213de704457865ff3b95d803f34: Neither blocks nor log segments found. Creating new log.
I20260806 22:53:01.432590 14465 log.cc:824] T 38b53c157d264a9d84725854c91d8697 P 60837b341f5d4b3eb7e6a53780b4abd3: Log is configured to *not* fsync() on all Append() calls
I20260806 22:53:01.432950 14466 log.cc:824] T 38b53c157d264a9d84725854c91d8697 P e9d32213de704457865ff3b95d803f34: Log is configured to *not* fsync() on all Append() calls
I20260806 22:53:01.433411 14465 tablet_bootstrap.cc:492] T 38b53c157d264a9d84725854c91d8697 P 60837b341f5d4b3eb7e6a53780b4abd3: No bootstrap required, opened a new log
I20260806 22:53:01.433482 14465 ts_tablet_manager.cc:1397] T 38b53c157d264a9d84725854c91d8697 P 60837b341f5d4b3eb7e6a53780b4abd3: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:53:01.433712 14466 tablet_bootstrap.cc:492] T 38b53c157d264a9d84725854c91d8697 P e9d32213de704457865ff3b95d803f34: No bootstrap required, opened a new log
I20260806 22:53:01.433763 14466 ts_tablet_manager.cc:1397] T 38b53c157d264a9d84725854c91d8697 P e9d32213de704457865ff3b95d803f34: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:53:01.434854 14465 raft_consensus.cc:348] T 38b53c157d264a9d84725854c91d8697 P 60837b341f5d4b3eb7e6a53780b4abd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60837b341f5d4b3eb7e6a53780b4abd3" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 38763 } } peers { permanent_uuid: "e9d32213de704457865ff3b95d803f34" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 46483 } } peers { permanent_uuid: "54b2b48fdddf4c6ea10ae001959e2ad3" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 43611 } }
I20260806 22:53:01.435007 14465 raft_consensus.cc:374] T 38b53c157d264a9d84725854c91d8697 P 60837b341f5d4b3eb7e6a53780b4abd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:53:01.435053 14465 raft_consensus.cc:729] T 38b53c157d264a9d84725854c91d8697 P 60837b341f5d4b3eb7e6a53780b4abd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60837b341f5d4b3eb7e6a53780b4abd3, State: Initialized, Role: FOLLOWER
I20260806 22:53:01.435086 14466 raft_consensus.cc:348] T 38b53c157d264a9d84725854c91d8697 P e9d32213de704457865ff3b95d803f34 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60837b341f5d4b3eb7e6a53780b4abd3" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 38763 } } peers { permanent_uuid: "e9d32213de704457865ff3b95d803f34" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 46483 } } peers { permanent_uuid: "54b2b48fdddf4c6ea10ae001959e2ad3" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 43611 } }
I20260806 22:53:01.435211 14465 consensus_queue.cc:260] T 38b53c157d264a9d84725854c91d8697 P 60837b341f5d4b3eb7e6a53780b4abd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60837b341f5d4b3eb7e6a53780b4abd3" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 38763 } } peers { permanent_uuid: "e9d32213de704457865ff3b95d803f34" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 46483 } } peers { permanent_uuid: "54b2b48fdddf4c6ea10ae001959e2ad3" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 43611 } }
I20260806 22:53:01.435299 14466 raft_consensus.cc:374] T 38b53c157d264a9d84725854c91d8697 P e9d32213de704457865ff3b95d803f34 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:53:01.435345 14466 raft_consensus.cc:729] T 38b53c157d264a9d84725854c91d8697 P e9d32213de704457865ff3b95d803f34 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9d32213de704457865ff3b95d803f34, State: Initialized, Role: FOLLOWER
I20260806 22:53:01.435467 14466 consensus_queue.cc:260] T 38b53c157d264a9d84725854c91d8697 P e9d32213de704457865ff3b95d803f34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60837b341f5d4b3eb7e6a53780b4abd3" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 38763 } } peers { permanent_uuid: "e9d32213de704457865ff3b95d803f34" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 46483 } } peers { permanent_uuid: "54b2b48fdddf4c6ea10ae001959e2ad3" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 43611 } }
I20260806 22:53:01.435518 14465 ts_tablet_manager.cc:1428] T 38b53c157d264a9d84725854c91d8697 P 60837b341f5d4b3eb7e6a53780b4abd3: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:53:01.435529 14454 heartbeater.cc:503] Master 127.28.218.254:35241 was elected leader, sending a full tablet report...
I20260806 22:53:01.435746 14466 ts_tablet_manager.cc:1428] T 38b53c157d264a9d84725854c91d8697 P e9d32213de704457865ff3b95d803f34: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:53:01.435775 14317 heartbeater.cc:503] Master 127.28.218.254:35241 was elected leader, sending a full tablet report...
W20260806 22:53:01.518136 14318 tablet.cc:2369] T 38b53c157d264a9d84725854c91d8697 P e9d32213de704457865ff3b95d803f34: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 22:53:01.635866 14173 tablet.cc:2369] T 38b53c157d264a9d84725854c91d8697 P 54b2b48fdddf4c6ea10ae001959e2ad3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 22:53:01.641124 14455 tablet.cc:2369] T 38b53c157d264a9d84725854c91d8697 P 60837b341f5d4b3eb7e6a53780b4abd3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:53:01.840706 14469 raft_consensus.cc:482] T 38b53c157d264a9d84725854c91d8697 P 60837b341f5d4b3eb7e6a53780b4abd3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:53:01.840842 14469 raft_consensus.cc:504] T 38b53c157d264a9d84725854c91d8697 P 60837b341f5d4b3eb7e6a53780b4abd3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60837b341f5d4b3eb7e6a53780b4abd3" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 38763 } } peers { permanent_uuid: "e9d32213de704457865ff3b95d803f34" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 46483 } } peers { permanent_uuid: "54b2b48fdddf4c6ea10ae001959e2ad3" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 43611 } }
I20260806 22:53:01.841254 14469 leader_election.cc:288] T 38b53c157d264a9d84725854c91d8697 P 60837b341f5d4b3eb7e6a53780b4abd3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9d32213de704457865ff3b95d803f34 (127.28.218.194:46483), 54b2b48fdddf4c6ea10ae001959e2ad3 (127.28.218.193:43611)
I20260806 22:53:01.845395 14456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:01.841480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.194:46483 (local address 127.28.218.195:38443)
0806 22:53:01.841569 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0806 22:53:01.841587 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 22:53:01.841671 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:53:01.841920 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:53:01.841921 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:53:01.841923 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:53:01.841994 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:53:01.841996 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:53:01.842977 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:53:01.842977 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:53:01.844404 (+ 1427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:53:01.844407 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:53:01.845130 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:53:01.845130 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:53:01.845215 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:01.845220 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:53:01.845270 (+ 50us) client_negotiation.cc:240] Negotiation successful
0806 22:53:01.845314 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66,"mutex_wait_us":42}
I20260806 22:53:01.845412 14463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:01.841601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.195:38443 (local address 127.28.218.194:46483)
0806 22:53:01.841778 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0806 22:53:01.841780 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:53:01.841792 (+ 12us) server_negotiation.cc:407] Connection header received
0806 22:53:01.841825 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:53:01.841826 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:53:01.841836 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:53:01.841906 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:53:01.842090 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:01.842879 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:01.844496 (+ 1617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:01.845026 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:01.845114 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:01.845123 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 22:53:01.845293 (+ 170us) server_negotiation.cc:299] Negotiation successful
0806 22:53:01.845339 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260806 22:53:01.845471 14473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:01.841706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.195:58269 (local address 127.28.218.193:43611)
0806 22:53:01.841809 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0806 22:53:01.841811 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:53:01.841845 (+ 34us) server_negotiation.cc:407] Connection header received
0806 22:53:01.841924 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:53:01.841925 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:53:01.841954 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:53:01.842023 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:53:01.842195 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:01.842907 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:01.844498 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:01.845144 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:01.845209 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:01.845293 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 22:53:01.845352 (+ 59us) server_negotiation.cc:299] Negotiation successful
0806 22:53:01.845393 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81,"thread_start_us":51,"threads_started":1}
I20260806 22:53:01.845508 14472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:01.841612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.193:43611 (local address 127.28.218.195:58269)
0806 22:53:01.841797 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0806 22:53:01.841805 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 22:53:01.841846 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:53:01.842056 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:53:01.842057 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:53:01.842059 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:53:01.842124 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:53:01.842125 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:53:01.842977 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:53:01.842977 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:53:01.844404 (+ 1427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:53:01.844407 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:53:01.845226 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:53:01.845227 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:53:01.845297 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:01.845300 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:53:01.845348 (+ 48us) client_negotiation.cc:240] Negotiation successful
0806 22:53:01.845376 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"mutex_wait_us":85,"thread_start_us":120,"threads_started":1}
I20260806 22:53:01.845800 14126 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38b53c157d264a9d84725854c91d8697" candidate_uuid: "60837b341f5d4b3eb7e6a53780b4abd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "54b2b48fdddf4c6ea10ae001959e2ad3" is_pre_election: true
I20260806 22:53:01.845808 14270 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38b53c157d264a9d84725854c91d8697" candidate_uuid: "60837b341f5d4b3eb7e6a53780b4abd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9d32213de704457865ff3b95d803f34" is_pre_election: true
I20260806 22:53:01.845958 14126 raft_consensus.cc:2454] T 38b53c157d264a9d84725854c91d8697 P 54b2b48fdddf4c6ea10ae001959e2ad3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60837b341f5d4b3eb7e6a53780b4abd3 in term 0.
I20260806 22:53:01.845990 14270 raft_consensus.cc:2454] T 38b53c157d264a9d84725854c91d8697 P e9d32213de704457865ff3b95d803f34 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60837b341f5d4b3eb7e6a53780b4abd3 in term 0.
I20260806 22:53:01.846196 14339 leader_election.cc:302] T 38b53c157d264a9d84725854c91d8697 P 60837b341f5d4b3eb7e6a53780b4abd3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54b2b48fdddf4c6ea10ae001959e2ad3, 60837b341f5d4b3eb7e6a53780b4abd3; no voters:
I20260806 22:53:01.846371 14469 raft_consensus.cc:2781] T 38b53c157d264a9d84725854c91d8697 P 60837b341f5d4b3eb7e6a53780b4abd3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:53:01.846442 14469 raft_consensus.cc:482] T 38b53c157d264a9d84725854c91d8697 P 60837b341f5d4b3eb7e6a53780b4abd3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:53:01.846478 14469 raft_consensus.cc:3037] T 38b53c157d264a9d84725854c91d8697 P 60837b341f5d4b3eb7e6a53780b4abd3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:53:01.847275 14469 raft_consensus.cc:504] T 38b53c157d264a9d84725854c91d8697 P 60837b341f5d4b3eb7e6a53780b4abd3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60837b341f5d4b3eb7e6a53780b4abd3" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 38763 } } peers { permanent_uuid: "e9d32213de704457865ff3b95d803f34" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 46483 } } peers { permanent_uuid: "54b2b48fdddf4c6ea10ae001959e2ad3" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 43611 } }
I20260806 22:53:01.847407 14469 leader_election.cc:288] T 38b53c157d264a9d84725854c91d8697 P 60837b341f5d4b3eb7e6a53780b4abd3 [CANDIDATE]: Term 1 election: Requested vote from peers e9d32213de704457865ff3b95d803f34 (127.28.218.194:46483), 54b2b48fdddf4c6ea10ae001959e2ad3 (127.28.218.193:43611)
I20260806 22:53:01.847635 14270 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38b53c157d264a9d84725854c91d8697" candidate_uuid: "60837b341f5d4b3eb7e6a53780b4abd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9d32213de704457865ff3b95d803f34"
I20260806 22:53:01.847666 14126 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38b53c157d264a9d84725854c91d8697" candidate_uuid: "60837b341f5d4b3eb7e6a53780b4abd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "54b2b48fdddf4c6ea10ae001959e2ad3"
I20260806 22:53:01.847725 14270 raft_consensus.cc:3037] T 38b53c157d264a9d84725854c91d8697 P e9d32213de704457865ff3b95d803f34 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:53:01.847772 14126 raft_consensus.cc:3037] T 38b53c157d264a9d84725854c91d8697 P 54b2b48fdddf4c6ea10ae001959e2ad3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:53:01.848424 14270 raft_consensus.cc:2454] T 38b53c157d264a9d84725854c91d8697 P e9d32213de704457865ff3b95d803f34 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60837b341f5d4b3eb7e6a53780b4abd3 in term 1.
I20260806 22:53:01.848618 14126 raft_consensus.cc:2454] T 38b53c157d264a9d84725854c91d8697 P 54b2b48fdddf4c6ea10ae001959e2ad3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60837b341f5d4b3eb7e6a53780b4abd3 in term 1.
I20260806 22:53:01.848641 14339 leader_election.cc:302] T 38b53c157d264a9d84725854c91d8697 P 60837b341f5d4b3eb7e6a53780b4abd3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60837b341f5d4b3eb7e6a53780b4abd3, e9d32213de704457865ff3b95d803f34; no voters:
I20260806 22:53:01.848732 14469 raft_consensus.cc:2781] T 38b53c157d264a9d84725854c91d8697 P 60837b341f5d4b3eb7e6a53780b4abd3 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:53:01.848878 14469 raft_consensus.cc:686] T 38b53c157d264a9d84725854c91d8697 P 60837b341f5d4b3eb7e6a53780b4abd3 [term 1 LEADER]: Becoming Leader. State: Replica: 60837b341f5d4b3eb7e6a53780b4abd3, State: Running, Role: LEADER
I20260806 22:53:01.848982 14469 consensus_queue.cc:237] T 38b53c157d264a9d84725854c91d8697 P 60837b341f5d4b3eb7e6a53780b4abd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60837b341f5d4b3eb7e6a53780b4abd3" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 38763 } } peers { permanent_uuid: "e9d32213de704457865ff3b95d803f34" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 46483 } } peers { permanent_uuid: "54b2b48fdddf4c6ea10ae001959e2ad3" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 43611 } }
I20260806 22:53:01.849798 13976 catalog_manager.cc:5495] T 38b53c157d264a9d84725854c91d8697 P 60837b341f5d4b3eb7e6a53780b4abd3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 60837b341f5d4b3eb7e6a53780b4abd3 (127.28.218.195). New cstate: current_term: 1 leader_uuid: "60837b341f5d4b3eb7e6a53780b4abd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60837b341f5d4b3eb7e6a53780b4abd3" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 38763 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9d32213de704457865ff3b95d803f34" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 46483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54b2b48fdddf4c6ea10ae001959e2ad3" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 43611 } health_report { overall_health: UNKNOWN } } }
I20260806 22:53:01.865725 14462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:01.863292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38836 (local address 127.28.218.195:38763)
0806 22:53:01.863358 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0806 22:53:01.863373 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 22:53:01.863383 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:53:01.863456 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:53:01.863457 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:53:01.863474 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:53:01.863535 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:53:01.863668 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:01.864157 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:01.864922 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:01.865146 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:01.865226 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:01.865393 (+ 167us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:53:01.865496 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:53:01.865590 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 22:53:01.865609 (+ 19us) server_negotiation.cc:299] Negotiation successful
0806 22:53:01.865626 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 22:53:01.867461 14126 raft_consensus.cc:1261] T 38b53c157d264a9d84725854c91d8697 P 54b2b48fdddf4c6ea10ae001959e2ad3 [term 1 FOLLOWER]: Refusing update from remote peer 60837b341f5d4b3eb7e6a53780b4abd3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:53:01.867460 14270 raft_consensus.cc:1261] T 38b53c157d264a9d84725854c91d8697 P e9d32213de704457865ff3b95d803f34 [term 1 FOLLOWER]: Refusing update from remote peer 60837b341f5d4b3eb7e6a53780b4abd3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:53:01.867799 14474 consensus_queue.cc:1035] T 38b53c157d264a9d84725854c91d8697 P 60837b341f5d4b3eb7e6a53780b4abd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9d32213de704457865ff3b95d803f34" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 46483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:53:01.867931 14469 consensus_queue.cc:1035] T 38b53c157d264a9d84725854c91d8697 P 60837b341f5d4b3eb7e6a53780b4abd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54b2b48fdddf4c6ea10ae001959e2ad3" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 43611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:53:01.869840 14477 mvcc.cc:205] Tried to move back new op lower bound from 7315688578526408704 to 7315688578454048768. Current Snapshot: MvccSnapshot[applied={T|T < 7315688578526408704}]
I20260806 22:53:01.872105 13976 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34378:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:53:01.872215 13976 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34378:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:53:01.872920 13976 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 97cb235f35f847ba97cf43d0dd94c0b5: Sending DeleteTablet for 3 replicas of tablet 38b53c157d264a9d84725854c91d8697
I20260806 22:53:01.873208 14386 tablet_service.cc:1514] Processing DeleteTablet for tablet 38b53c157d264a9d84725854c91d8697 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:53:01 UTC) from {username='kudu', principal='kudu/127.28.218.254@KRBTEST.COM'} at 127.0.0.1:38828
I20260806 22:53:01.873258 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 14015
I20260806 22:53:01.873318 14249 tablet_service.cc:1514] Processing DeleteTablet for tablet 38b53c157d264a9d84725854c91d8697 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:53:01 UTC) from {username='kudu', principal='kudu/127.28.218.254@KRBTEST.COM'} at 127.0.0.1:51004
I20260806 22:53:01.873381 14105 tablet_service.cc:1514] Processing DeleteTablet for tablet 38b53c157d264a9d84725854c91d8697 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:53:01 UTC) from {username='kudu', principal='kudu/127.28.218.254@KRBTEST.COM'} at 127.0.0.1:36068
I20260806 22:53:01.874788 14487 tablet_replica.cc:330] T 38b53c157d264a9d84725854c91d8697 P e9d32213de704457865ff3b95d803f34: stopping tablet replica
I20260806 22:53:01.874873 14487 raft_consensus.cc:2229] T 38b53c157d264a9d84725854c91d8697 P e9d32213de704457865ff3b95d803f34 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:53:01.874958 14487 raft_consensus.cc:2258] T 38b53c157d264a9d84725854c91d8697 P e9d32213de704457865ff3b95d803f34 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:53:01.875519 14487 ts_tablet_manager.cc:1905] T 38b53c157d264a9d84725854c91d8697 P e9d32213de704457865ff3b95d803f34: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:53:01.877712 14487 ts_tablet_manager.cc:1918] T 38b53c157d264a9d84725854c91d8697 P e9d32213de704457865ff3b95d803f34: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:53:01.877785 14487 log.cc:1196] T 38b53c157d264a9d84725854c91d8697 P e9d32213de704457865ff3b95d803f34: Deleting WAL directory at /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-1/wal/wals/38b53c157d264a9d84725854c91d8697
I20260806 22:53:01.878089 14487 ts_tablet_manager.cc:1939] T 38b53c157d264a9d84725854c91d8697 P e9d32213de704457865ff3b95d803f34: Deleting consensus metadata
I20260806 22:53:01.878643 13962 catalog_manager.cc:4841] TS e9d32213de704457865ff3b95d803f34 (127.28.218.194:46483): tablet 38b53c157d264a9d84725854c91d8697 (table test-table [id=20364c58117a4784b3948a88b2707192]) successfully deleted
I20260806 22:53:01.880184 14486 tablet_replica.cc:330] T 38b53c157d264a9d84725854c91d8697 P 60837b341f5d4b3eb7e6a53780b4abd3: stopping tablet replica
I20260806 22:53:01.880303 14486 raft_consensus.cc:2229] T 38b53c157d264a9d84725854c91d8697 P 60837b341f5d4b3eb7e6a53780b4abd3 [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:53:01.880427 14486 raft_consensus.cc:2258] T 38b53c157d264a9d84725854c91d8697 P 60837b341f5d4b3eb7e6a53780b4abd3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:53:01.880785 14486 ts_tablet_manager.cc:1905] T 38b53c157d264a9d84725854c91d8697 P 60837b341f5d4b3eb7e6a53780b4abd3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:53:01.882313 14486 ts_tablet_manager.cc:1918] T 38b53c157d264a9d84725854c91d8697 P 60837b341f5d4b3eb7e6a53780b4abd3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:53:01.882406 14486 log.cc:1196] T 38b53c157d264a9d84725854c91d8697 P 60837b341f5d4b3eb7e6a53780b4abd3: Deleting WAL directory at /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-2/wal/wals/38b53c157d264a9d84725854c91d8697
I20260806 22:53:01.882661 14486 ts_tablet_manager.cc:1939] T 38b53c157d264a9d84725854c91d8697 P 60837b341f5d4b3eb7e6a53780b4abd3: Deleting consensus metadata
I20260806 22:53:01.883023 13961 catalog_manager.cc:4841] TS 60837b341f5d4b3eb7e6a53780b4abd3 (127.28.218.195:38763): tablet 38b53c157d264a9d84725854c91d8697 (table test-table [id=20364c58117a4784b3948a88b2707192]) successfully deleted
W20260806 22:53:01.884827 13962 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.28.218.193:43611 (error 108)
I20260806 22:53:01.885217 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 14178
I20260806 22:53:01.885927 14461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:01.885359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.193:43611 (local address 127.0.0.1:36072)
0806 22:53:01.885845 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0806 22:53:01.885892 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.193:43611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":465}
W20260806 22:53:01.887100 13962 catalog_manager.cc:4584] TS 54b2b48fdddf4c6ea10ae001959e2ad3 (127.28.218.193:43611): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 38b53c157d264a9d84725854c91d8697: Network error: Client connection negotiation failed: client connection to 127.28.218.193:43611: connect: Connection refused (error 111)
I20260806 22:53:01.896348 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 14320
I20260806 22:53:01.902300 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 13941
I20260806 22:53:21.908723 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.218.254:35241
--webserver_interface=127.28.218.254
--webserver_port=35267
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab
--principal=kudu/127.28.218.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.218.212:40995
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.218.254:35241
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:53:21.992102 14490 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:53:21.992306 14490 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:53:21.992328 14490 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:53:21.993486 14490 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 22:53:21.993530 14490 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 22:53:21.993543 14490 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 22:53:21.993556 14490 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:53:21.993603 14490 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:53:21.993621 14490 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:53:21.993642 14490 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:53:21.993667 14490 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:53:21.995090 14490 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:40995
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.28.218.254:35241
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab
--principal=kudu/127.28.218.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.218.254:35241
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.218.254
--webserver_port=35267
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14490
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:53:21.995380 14490 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:53:21.995626 14490 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 22:53:22.000476 14490 server_base.cc:1034] running on GCE node
W20260806 22:53:22.000572 14495 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:53:22.000528 14496 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:53:22.000717 14498 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:53:22.000948 14490 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:53:22.001232 14490 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:53:22.002411 14490 hybrid_clock.cc:648] HybridClock initialized: now 1786056802002405 us; error 36 us; skew 500 ppm
Aug 06 22:53:22 dist-test-slave-1xrh krb5kdc[13362](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056802, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:53:22.010382 14490 init.cc:377] Logged in from keytab as kudu/127.28.218.254@KRBTEST.COM (short username kudu)
I20260806 22:53:22.010823 14490 webserver.cc:460] Webserver started at http://127.28.218.254:35267/ using document root <none> and password file <none>
I20260806 22:53:22.011046 14490 fs_manager.cc:359] Metadata directory not provided
I20260806 22:53:22.011114 14490 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:53:22.012457 14490 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:53:22.013118 14505 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:53:22.013278 14490 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:53:22.013378 14490 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/master-0/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/master-0/wal
uuid: "97cb235f35f847ba97cf43d0dd94c0b5"
format_stamp: "Formatted at 2026-08-06 22:52:40 on dist-test-slave-1xrh"
server_key: "0b6344b64d49e81ccfbc16cbdecc3682"
server_key_iv: "8dff30e4a2f249b76ea801a7289e364f"
server_key_version: "encryptionkey@0"
I20260806 22:53:22.013725 14490 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:53:22.029157 14490 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:53:22.030892 14490 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:53:22.031051 14490 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:53:22.034894 14490 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.254:35241
I20260806 22:53:22.034955 14560 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.254:35241 every 8 connection(s)
I20260806 22:53:22.035369 14490 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
I20260806 22:53:22.036743 14562 sys_catalog.cc:263] Verifying existing consensus state
I20260806 22:53:22.037390 14562 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97cb235f35f847ba97cf43d0dd94c0b5: Bootstrap starting.
I20260806 22:53:22.044414 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 14490
I20260806 22:53:22.044688 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.218.193:43611
--local_ip_for_outbound_sockets=127.28.218.193
--tserver_master_addrs=127.28.218.254:35241
--webserver_port=35237
--webserver_interface=127.28.218.193
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.193.keytab
--principal=kudu/127.28.218.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.218.212:40995
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:53:22.045790 14562 log.cc:824] T 00000000000000000000000000000000 P 97cb235f35f847ba97cf43d0dd94c0b5: Log is configured to *not* fsync() on all Append() calls
I20260806 22:53:22.048385 14562 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97cb235f35f847ba97cf43d0dd94c0b5: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 22:53:22.048681 14562 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97cb235f35f847ba97cf43d0dd94c0b5: Bootstrap complete.
I20260806 22:53:22.050887 14562 raft_consensus.cc:348] T 00000000000000000000000000000000 P 97cb235f35f847ba97cf43d0dd94c0b5 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97cb235f35f847ba97cf43d0dd94c0b5" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 35241 } }
I20260806 22:53:22.051120 14562 raft_consensus.cc:729] T 00000000000000000000000000000000 P 97cb235f35f847ba97cf43d0dd94c0b5 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97cb235f35f847ba97cf43d0dd94c0b5, State: Initialized, Role: FOLLOWER
I20260806 22:53:22.051316 14562 consensus_queue.cc:260] T 00000000000000000000000000000000 P 97cb235f35f847ba97cf43d0dd94c0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97cb235f35f847ba97cf43d0dd94c0b5" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 35241 } }
I20260806 22:53:22.051411 14562 raft_consensus.cc:388] T 00000000000000000000000000000000 P 97cb235f35f847ba97cf43d0dd94c0b5 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:53:22.051463 14562 raft_consensus.cc:482] T 00000000000000000000000000000000 P 97cb235f35f847ba97cf43d0dd94c0b5 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:53:22.051522 14562 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 97cb235f35f847ba97cf43d0dd94c0b5 [term 2 FOLLOWER]: Advancing to term 3
I20260806 22:53:22.052495 14562 raft_consensus.cc:504] T 00000000000000000000000000000000 P 97cb235f35f847ba97cf43d0dd94c0b5 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97cb235f35f847ba97cf43d0dd94c0b5" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 35241 } }
I20260806 22:53:22.052611 14562 leader_election.cc:302] T 00000000000000000000000000000000 P 97cb235f35f847ba97cf43d0dd94c0b5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 97cb235f35f847ba97cf43d0dd94c0b5; no voters:
I20260806 22:53:22.052802 14562 leader_election.cc:288] T 00000000000000000000000000000000 P 97cb235f35f847ba97cf43d0dd94c0b5 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260806 22:53:22.052841 14567 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 97cb235f35f847ba97cf43d0dd94c0b5 [term 3 FOLLOWER]: Leader election won for term 3
I20260806 22:53:22.052959 14567 raft_consensus.cc:686] T 00000000000000000000000000000000 P 97cb235f35f847ba97cf43d0dd94c0b5 [term 3 LEADER]: Becoming Leader. State: Replica: 97cb235f35f847ba97cf43d0dd94c0b5, State: Running, Role: LEADER
I20260806 22:53:22.053076 14562 sys_catalog.cc:564] T 00000000000000000000000000000000 P 97cb235f35f847ba97cf43d0dd94c0b5 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:53:22.053063 14567 consensus_queue.cc:237] T 00000000000000000000000000000000 P 97cb235f35f847ba97cf43d0dd94c0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97cb235f35f847ba97cf43d0dd94c0b5" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 35241 } }
I20260806 22:53:22.053448 14568 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97cb235f35f847ba97cf43d0dd94c0b5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "97cb235f35f847ba97cf43d0dd94c0b5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97cb235f35f847ba97cf43d0dd94c0b5" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 35241 } } }
I20260806 22:53:22.053449 14569 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97cb235f35f847ba97cf43d0dd94c0b5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 97cb235f35f847ba97cf43d0dd94c0b5. Latest consensus state: current_term: 3 leader_uuid: "97cb235f35f847ba97cf43d0dd94c0b5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97cb235f35f847ba97cf43d0dd94c0b5" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 35241 } } }
I20260806 22:53:22.053650 14568 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97cb235f35f847ba97cf43d0dd94c0b5 [sys.catalog]: This master's current role is: LEADER
I20260806 22:53:22.053669 14569 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97cb235f35f847ba97cf43d0dd94c0b5 [sys.catalog]: This master's current role is: LEADER
I20260806 22:53:22.054215 14578 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:53:22.055006 14578 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:53:22.055107 14578 catalog_manager.cc:1259] Loaded cluster ID: 3c7f71f6744b4458a6f069c02919cdd6
I20260806 22:53:22.055157 14578 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:53:22.056334 14578 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:53:22.056526 14578 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 97cb235f35f847ba97cf43d0dd94c0b5: Loaded TSK: 0
I20260806 22:53:22.056577 14578 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 97cb235f35f847ba97cf43d0dd94c0b5: Loaded TSK: 1
F20260806 22:53:22.056807 14578 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f38eae909ad google::LogMessage::Fail() at ??:0
@ 0x7f38eae9388a google::LogMessage::SendToLog() at ??:0
@ 0x7f38eae904de google::LogMessage::Flush() at ??:0
@ 0x7f38eae91a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f38ed5fd104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f38ed5fad41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f38f2dea1cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f38f2dea48e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f38f2dea594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f38f2db0c48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f38f2dbaad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f38ecd5e0d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f38ecd53954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f38edbb76db start_thread at ??:0
@ 0x7f38e998e61f clone at ??:0
W20260806 22:53:22.136771 14566 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:53:22.137005 14566 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:53:22.137046 14566 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:53:22.138146 14566 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:53:22.138208 14566 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:53:22.138252 14566 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.218.193
I20260806 22:53:22.139756 14566 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:40995
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.193.keytab
--principal=kudu/127.28.218.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.218.193:43611
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.218.193
--webserver_port=35237
--tserver_master_addrs=127.28.218.254:35241
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14566
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.218.193
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:53:22.140022 14566 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:53:22.140264 14566 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:53:22.145125 14606 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:53:22.145383 14566 server_base.cc:1034] running on GCE node
W20260806 22:53:22.145530 14609 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:53:22.145176 14607 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:53:22.145782 14566 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:53:22.146047 14566 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:53:22.147173 14566 hybrid_clock.cc:648] HybridClock initialized: now 1786056802147162 us; error 27 us; skew 500 ppm
Aug 06 22:53:22 dist-test-slave-1xrh krb5kdc[13362](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056802, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:53:22.149715 14566 init.cc:377] Logged in from keytab as kudu/127.28.218.193@KRBTEST.COM (short username kudu)
I20260806 22:53:22.150183 14566 webserver.cc:460] Webserver started at http://127.28.218.193:35237/ using document root <none> and password file <none>
I20260806 22:53:22.150452 14566 fs_manager.cc:359] Metadata directory not provided
I20260806 22:53:22.150518 14566 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:53:22.151881 14566 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.000s
I20260806 22:53:22.152539 14616 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:53:22.152712 14566 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:53:22.152803 14566 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-0/wal
uuid: "54b2b48fdddf4c6ea10ae001959e2ad3"
format_stamp: "Formatted at 2026-08-06 22:52:40 on dist-test-slave-1xrh"
server_key: "3a5eced6bd4526f6244d49e99c177928"
server_key_iv: "6dc9cd2527d76c346819058f2dfbdd57"
server_key_version: "encryptionkey@0"
I20260806 22:53:22.153151 14566 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:53:22.166997 14566 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:53:22.169203 14566 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:53:22.169348 14566 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:53:22.169565 14566 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:53:22.169983 14623 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 22:53:22.171980 14566 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 22:53:22.172099 14566 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260806 22:53:22.172197 14566 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 22:53:22.172922 14566 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 22:53:22.172994 14566 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260806 22:53:22.173045 14623 tablet_bootstrap.cc:492] T 38b53c157d264a9d84725854c91d8697 P 54b2b48fdddf4c6ea10ae001959e2ad3: Bootstrap starting.
I20260806 22:53:22.180119 14566 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.193:43611
I20260806 22:53:22.180160 14739 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.193:43611 every 8 connection(s)
I20260806 22:53:22.180523 14566 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-0/data/info.pb
I20260806 22:53:22.180874 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 14566
I20260806 22:53:22.181173 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.218.194:46483
--local_ip_for_outbound_sockets=127.28.218.194
--tserver_master_addrs=127.28.218.254:35241
--webserver_port=42403
--webserver_interface=127.28.218.194
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.194.keytab
--principal=kudu/127.28.218.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.218.212:40995
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:53:22.183893 14623 log.cc:824] T 38b53c157d264a9d84725854c91d8697 P 54b2b48fdddf4c6ea10ae001959e2ad3: Log is configured to *not* fsync() on all Append() calls
I20260806 22:53:22.188745 14623 tablet_bootstrap.cc:492] T 38b53c157d264a9d84725854c91d8697 P 54b2b48fdddf4c6ea10ae001959e2ad3: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 22:53:22.189057 14623 tablet_bootstrap.cc:492] T 38b53c157d264a9d84725854c91d8697 P 54b2b48fdddf4c6ea10ae001959e2ad3: Bootstrap complete.
I20260806 22:53:22.189985 14623 ts_tablet_manager.cc:1397] T 38b53c157d264a9d84725854c91d8697 P 54b2b48fdddf4c6ea10ae001959e2ad3: Time spent bootstrapping tablet: real 0.017s user 0.004s sys 0.009s
I20260806 22:53:22.190948 14623 raft_consensus.cc:348] T 38b53c157d264a9d84725854c91d8697 P 54b2b48fdddf4c6ea10ae001959e2ad3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60837b341f5d4b3eb7e6a53780b4abd3" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 38763 } } peers { permanent_uuid: "e9d32213de704457865ff3b95d803f34" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 46483 } } peers { permanent_uuid: "54b2b48fdddf4c6ea10ae001959e2ad3" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 43611 } }
I20260806 22:53:22.191242 14623 raft_consensus.cc:729] T 38b53c157d264a9d84725854c91d8697 P 54b2b48fdddf4c6ea10ae001959e2ad3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54b2b48fdddf4c6ea10ae001959e2ad3, State: Initialized, Role: FOLLOWER
I20260806 22:53:22.191378 14623 consensus_queue.cc:260] T 38b53c157d264a9d84725854c91d8697 P 54b2b48fdddf4c6ea10ae001959e2ad3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60837b341f5d4b3eb7e6a53780b4abd3" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 38763 } } peers { permanent_uuid: "e9d32213de704457865ff3b95d803f34" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 46483 } } peers { permanent_uuid: "54b2b48fdddf4c6ea10ae001959e2ad3" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 43611 } }
I20260806 22:53:22.191664 14623 ts_tablet_manager.cc:1428] T 38b53c157d264a9d84725854c91d8697 P 54b2b48fdddf4c6ea10ae001959e2ad3: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
W20260806 22:53:22.278930 14743 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:53:22.183868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:35241 (local address 127.28.218.193:54483)
0806 22:53:22.184135 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0806 22:53:22.184178 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 22:53:22.184682 (+ 504us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:53:22.278723 (+ 94041us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:35241: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":93,"threads_started":1}
W20260806 22:53:22.279206 14740 heartbeater.cc:644] Failed to heartbeat to 127.28.218.254:35241 (0 consecutive failures): Network error: Failed to ping master at 127.28.218.254:35241: Client connection negotiation failed: client connection to 127.28.218.254:35241: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 22:53:22.279525 14743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:22.279406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:35241 (local address 127.28.218.193:40581)
0806 22:53:22.279461 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0806 22:53:22.279497 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:35241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260806 22:53:22.279970 14743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:22.279885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:35241 (local address 127.28.218.193:41687)
0806 22:53:22.279929 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0806 22:53:22.279956 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:35241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260806 22:53:22.280094 14740 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 22:53:22.286232 14742 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:53:22.286438 14742 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:53:22.286489 14742 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:53:22.287698 14742 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:53:22.287753 14742 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:53:22.287802 14742 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.218.194
I20260806 22:53:22.289218 14742 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:40995
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.194.keytab
--principal=kudu/127.28.218.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.218.194:46483
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.218.194
--webserver_port=42403
--tserver_master_addrs=127.28.218.254:35241
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14742
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.218.194
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:53:22.289476 14742 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:53:22.289685 14742 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:53:22.294447 14751 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:53:22.294486 14750 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:53:22.294489 14754 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:53:22.294656 14742 server_base.cc:1034] running on GCE node
I20260806 22:53:22.294845 14742 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:53:22.295089 14742 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:53:22.296268 14742 hybrid_clock.cc:648] HybridClock initialized: now 1786056802296254 us; error 28 us; skew 500 ppm
Aug 06 22:53:22 dist-test-slave-1xrh krb5kdc[13362](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056802, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:53:22.298595 14742 init.cc:377] Logged in from keytab as kudu/127.28.218.194@KRBTEST.COM (short username kudu)
I20260806 22:53:22.298996 14742 webserver.cc:460] Webserver started at http://127.28.218.194:42403/ using document root <none> and password file <none>
I20260806 22:53:22.299291 14742 fs_manager.cc:359] Metadata directory not provided
I20260806 22:53:22.299366 14742 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:53:22.300621 14742 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260806 22:53:22.301188 14761 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:53:22.301345 14742 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 22:53:22.301429 14742 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-1/wal
uuid: "e9d32213de704457865ff3b95d803f34"
format_stamp: "Formatted at 2026-08-06 22:52:40 on dist-test-slave-1xrh"
server_key: "7de9e2e87fdbfc25d45f21e606f19973"
server_key_iv: "72954717981183c76dfab68b323d4e36"
server_key_version: "encryptionkey@0"
I20260806 22:53:22.301728 14742 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:53:22.314179 14742 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:53:22.315986 14742 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:53:22.316149 14742 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:53:22.316363 14742 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:53:22.316649 14742 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:53:22.316699 14742 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:53:22.316733 14742 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:53:22.316761 14742 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:53:22.322443 14742 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.194:46483
I20260806 22:53:22.322511 14880 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.194:46483 every 8 connection(s)
I20260806 22:53:22.322829 14742 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-1/data/info.pb
I20260806 22:53:22.323793 14884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:22.323454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:35241 (local address 127.28.218.194:59421)
0806 22:53:22.323600 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0806 22:53:22.323663 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:35241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":74,"threads_started":1}
W20260806 22:53:22.324162 14881 heartbeater.cc:644] Failed to heartbeat to 127.28.218.254:35241 (0 consecutive failures): Network error: Failed to ping master at 127.28.218.254:35241: Client connection negotiation failed: client connection to 127.28.218.254:35241: connect: Connection refused (error 111)
I20260806 22:53:22.324414 14884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:22.324337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:35241 (local address 127.28.218.194:41535)
0806 22:53:22.324375 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0806 22:53:22.324399 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:35241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260806 22:53:22.324653 14884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:22.324606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:35241 (local address 127.28.218.194:53789)
0806 22:53:22.324625 (+ 19us) negotiation.cc:107] Waiting for socket to connect
0806 22:53:22.324642 (+ 17us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:35241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":12}
W20260806 22:53:22.324774 14881 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 22:53:22.327742 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 14742
I20260806 22:53:22.328033 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.218.195:38763
--local_ip_for_outbound_sockets=127.28.218.195
--tserver_master_addrs=127.28.218.254:35241
--webserver_port=44067
--webserver_interface=127.28.218.195
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.195.keytab
--principal=kudu/127.28.218.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.218.212:40995
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:53:22.410091 14885 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:53:22.410279 14885 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:53:22.410301 14885 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:53:22.411413 14885 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:53:22.411451 14885 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:53:22.411476 14885 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.218.195
I20260806 22:53:22.412771 14885 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:40995
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.195.keytab
--principal=kudu/127.28.218.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.218.195:38763
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.218.195
--webserver_port=44067
--tserver_master_addrs=127.28.218.254:35241
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14885
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.218.195
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:53:22.412951 14885 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:53:22.413110 14885 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:53:22.417898 14893 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:53:22.417912 14890 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:53:22.418174 14885 server_base.cc:1034] running on GCE node
W20260806 22:53:22.417912 14891 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:53:22.418414 14885 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:53:22.418629 14885 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:53:22.419770 14885 hybrid_clock.cc:648] HybridClock initialized: now 1786056802419760 us; error 29 us; skew 500 ppm
Aug 06 22:53:22 dist-test-slave-1xrh krb5kdc[13362](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056802, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:53:22.422202 14885 init.cc:377] Logged in from keytab as kudu/127.28.218.195@KRBTEST.COM (short username kudu)
I20260806 22:53:22.422626 14885 webserver.cc:460] Webserver started at http://127.28.218.195:44067/ using document root <none> and password file <none>
I20260806 22:53:22.422829 14885 fs_manager.cc:359] Metadata directory not provided
I20260806 22:53:22.422894 14885 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:53:22.424309 14885 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:53:22.424875 14900 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:53:22.425052 14885 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:53:22.425140 14885 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-2/wal
uuid: "60837b341f5d4b3eb7e6a53780b4abd3"
format_stamp: "Formatted at 2026-08-06 22:52:40 on dist-test-slave-1xrh"
server_key: "4b2e8dedab4651ff535f40daf801b17c"
server_key_iv: "dcc9a875e77dc9e6a9718036fd237485"
server_key_version: "encryptionkey@0"
I20260806 22:53:22.425441 14885 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:53:22.434274 14885 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:53:22.436339 14885 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:53:22.436496 14885 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:53:22.436705 14885 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:53:22.436986 14885 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:53:22.437036 14885 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:53:22.437075 14885 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:53:22.437109 14885 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:53:22.443272 15018 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.195:38763 every 8 connection(s)
I20260806 22:53:22.443377 14885 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.195:38763
I20260806 22:53:22.443912 14885 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056586788339-29547-0/minicluster-data/ts-2/data/info.pb
I20260806 22:53:22.444981 15021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:22.444626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:35241 (local address 127.28.218.195:42055)
0806 22:53:22.444783 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0806 22:53:22.444851 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:35241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":73,"threads_started":1}
W20260806 22:53:22.445358 15019 heartbeater.cc:644] Failed to heartbeat to 127.28.218.254:35241 (0 consecutive failures): Network error: Failed to ping master at 127.28.218.254:35241: Client connection negotiation failed: client connection to 127.28.218.254:35241: connect: Connection refused (error 111)
I20260806 22:53:22.445642 15021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:22.445539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:35241 (local address 127.28.218.195:49535)
0806 22:53:22.445589 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0806 22:53:22.445621 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:35241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260806 22:53:22.445951 15021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:22.445864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:35241 (local address 127.28.218.195:35925)
0806 22:53:22.445909 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0806 22:53:22.445932 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:35241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260806 22:53:22.446060 15019 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 22:53:22.453361 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 14885
I20260806 22:53:23.280658 15023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:23.280414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:35241 (local address 127.28.218.193:51771)
0806 22:53:23.280548 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0806 22:53:23.280589 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:35241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":62,"threads_started":1}
I20260806 22:53:23.325241 15024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:23.325047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:35241 (local address 127.28.218.194:39347)
0806 22:53:23.325159 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0806 22:53:23.325190 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:35241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":57,"threads_started":1}
I20260806 22:53:23.446655 15025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:23.446440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:35241 (local address 127.28.218.195:53893)
0806 22:53:23.446567 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0806 22:53:23.446603 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:35241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":63,"threads_started":1}
I20260806 22:53:23.843489 15026 raft_consensus.cc:482] T 38b53c157d264a9d84725854c91d8697 P 54b2b48fdddf4c6ea10ae001959e2ad3 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:53:23.843626 15026 raft_consensus.cc:504] T 38b53c157d264a9d84725854c91d8697 P 54b2b48fdddf4c6ea10ae001959e2ad3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60837b341f5d4b3eb7e6a53780b4abd3" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 38763 } } peers { permanent_uuid: "e9d32213de704457865ff3b95d803f34" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 46483 } } peers { permanent_uuid: "54b2b48fdddf4c6ea10ae001959e2ad3" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 43611 } }
I20260806 22:53:23.844058 15026 leader_election.cc:288] T 38b53c157d264a9d84725854c91d8697 P 54b2b48fdddf4c6ea10ae001959e2ad3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 60837b341f5d4b3eb7e6a53780b4abd3 (127.28.218.195:38763), e9d32213de704457865ff3b95d803f34 (127.28.218.194:46483)
Aug 06 22:53:23 dist-test-slave-1xrh krb5kdc[13362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056802, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.193@KRBTEST.COM for kudu/127.28.218.195@KRBTEST.COM
Aug 06 22:53:23 dist-test-slave-1xrh krb5kdc[13362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056802, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.193@KRBTEST.COM for kudu/127.28.218.194@KRBTEST.COM
I20260806 22:53:23.856102 15030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:23.844427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.193:34119 (local address 127.28.218.194:46483)
0806 22:53:23.844608 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0806 22:53:23.844621 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 22:53:23.844651 (+ 30us) server_negotiation.cc:407] Connection header received
0806 22:53:23.845234 (+ 583us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:53:23.845237 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:53:23.845298 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:53:23.845463 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:53:23.846180 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:23.847077 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:23.848682 (+ 1605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:23.849329 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:23.849448 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:23.852671 (+ 3223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:53:23.852675 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:53:23.852676 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:53:23.852688 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:53:23.854366 (+ 1678us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:53:23.854786 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:53:23.854788 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:53:23.854788 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:53:23.854875 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:53:23.855163 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:53:23.855164 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:53:23.855164 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:53:23.855438 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:53:23.855561 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0806 22:53:23.855850 (+ 289us) server_negotiation.cc:299] Negotiation successful
0806 22:53:23.855983 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":58,"threads_started":1}
I20260806 22:53:23.856101 15028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:23.844262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.195:38763 (local address 127.28.218.193:37661)
0806 22:53:23.844436 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0806 22:53:23.844447 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:53:23.844525 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:53:23.845604 (+ 1079us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:53:23.845605 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:53:23.845863 (+ 258us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:53:23.846033 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:53:23.846038 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:53:23.847239 (+ 1201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:53:23.847240 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:53:23.848585 (+ 1345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:53:23.848588 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:53:23.849434 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:53:23.849435 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:53:23.849582 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:23.850233 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:53:23.850245 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:53:23.852191 (+ 1946us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:53:23.854363 (+ 2172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:53:23.854372 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:53:23.854374 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:53:23.854667 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:53:23.854937 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:53:23.854938 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:53:23.854939 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:53:23.855069 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:53:23.855566 (+ 497us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:53:23.855567 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:53:23.855677 (+ 110us) client_negotiation.cc:769] Sending connection context
0806 22:53:23.855785 (+ 108us) client_negotiation.cc:240] Negotiation successful
0806 22:53:23.856004 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"mutex_wait_us":84,"thread_start_us":85,"threads_started":1}
I20260806 22:53:23.856143 15029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:23.844430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.193:37661 (local address 127.28.218.195:38763)
0806 22:53:23.844608 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0806 22:53:23.844621 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 22:53:23.844653 (+ 32us) server_negotiation.cc:407] Connection header received
0806 22:53:23.845234 (+ 581us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:53:23.845237 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:53:23.845302 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:53:23.845479 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:53:23.846168 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:23.847084 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:23.848681 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:23.849328 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:23.849464 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:23.852314 (+ 2850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:53:23.852318 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:53:23.852319 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:53:23.852333 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:53:23.854273 (+ 1940us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:53:23.854752 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:53:23.854756 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:53:23.854756 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:53:23.854865 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:53:23.855156 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:53:23.855157 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:53:23.855157 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:53:23.855485 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:53:23.855572 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 22:53:23.855857 (+ 285us) server_negotiation.cc:299] Negotiation successful
0806 22:53:23.856021 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":57,"threads_started":1}
I20260806 22:53:23.856225 15027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:23.844262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.194:46483 (local address 127.28.218.193:34119)
0806 22:53:23.844582 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0806 22:53:23.844589 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0806 22:53:23.844614 (+ 25us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:53:23.845514 (+ 900us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:53:23.845517 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:53:23.845836 (+ 319us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:53:23.846032 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:53:23.846038 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:53:23.847257 (+ 1219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:53:23.847258 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:53:23.848585 (+ 1327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:53:23.848588 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:53:23.849433 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:53:23.849434 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:53:23.849582 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:23.850233 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:53:23.850245 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:53:23.852562 (+ 2317us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:53:23.854465 (+ 1903us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:53:23.854466 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:53:23.854467 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:53:23.854680 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:53:23.854968 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:53:23.854973 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:53:23.854973 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:53:23.855070 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:53:23.855544 (+ 474us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:53:23.855552 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:53:23.855677 (+ 125us) client_negotiation.cc:769] Sending connection context
0806 22:53:23.855786 (+ 109us) client_negotiation.cc:240] Negotiation successful
0806 22:53:23.856088 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":313,"mutex_wait_us":231,"thread_start_us":66,"threads_started":1}
W20260806 22:53:23.856967 14619 leader_election.cc:341] T 38b53c157d264a9d84725854c91d8697 P 54b2b48fdddf4c6ea10ae001959e2ad3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 60837b341f5d4b3eb7e6a53780b4abd3 (127.28.218.195:38763): Not found: Tablet not found: 38b53c157d264a9d84725854c91d8697
W20260806 22:53:23.857445 14620 leader_election.cc:341] T 38b53c157d264a9d84725854c91d8697 P 54b2b48fdddf4c6ea10ae001959e2ad3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e9d32213de704457865ff3b95d803f34 (127.28.218.194:46483): Not found: Tablet not found: 38b53c157d264a9d84725854c91d8697
I20260806 22:53:23.857512 14620 leader_election.cc:302] T 38b53c157d264a9d84725854c91d8697 P 54b2b48fdddf4c6ea10ae001959e2ad3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54b2b48fdddf4c6ea10ae001959e2ad3; no voters: 60837b341f5d4b3eb7e6a53780b4abd3, e9d32213de704457865ff3b95d803f34
I20260806 22:53:23.857658 15026 raft_consensus.cc:2726] T 38b53c157d264a9d84725854c91d8697 P 54b2b48fdddf4c6ea10ae001959e2ad3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 22:53:24.281347 15027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:24.281181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:35241 (local address 127.28.218.193:50421)
0806 22:53:24.281264 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0806 22:53:24.281296 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:35241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260806 22:53:24.325968 15031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:24.325688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:35241 (local address 127.28.218.194:35097)
0806 22:53:24.325838 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0806 22:53:24.325897 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:35241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":84,"threads_started":1}
I20260806 22:53:24.447481 15032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:24.447260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:35241 (local address 127.28.218.195:40775)
0806 22:53:24.447382 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0806 22:53:24.447425 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:35241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":63,"threads_started":1}
I20260806 22:53:25.282058 15033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:25.281845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:35241 (local address 127.28.218.193:56513)
0806 22:53:25.281972 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0806 22:53:25.282009 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:35241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":66,"threads_started":1}
I20260806 22:53:25.326655 15034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:25.326400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:35241 (local address 127.28.218.194:49913)
0806 22:53:25.326539 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0806 22:53:25.326593 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:35241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":78,"threads_started":1}
I20260806 22:53:25.448091 15035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:25.447894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:35241 (local address 127.28.218.195:59301)
0806 22:53:25.448020 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0806 22:53:25.448052 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:35241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":69,"threads_started":1}
I20260806 22:53:26.017022 15036 raft_consensus.cc:482] T 38b53c157d264a9d84725854c91d8697 P 54b2b48fdddf4c6ea10ae001959e2ad3 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:53:26.017143 15036 raft_consensus.cc:504] T 38b53c157d264a9d84725854c91d8697 P 54b2b48fdddf4c6ea10ae001959e2ad3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60837b341f5d4b3eb7e6a53780b4abd3" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 38763 } } peers { permanent_uuid: "e9d32213de704457865ff3b95d803f34" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 46483 } } peers { permanent_uuid: "54b2b48fdddf4c6ea10ae001959e2ad3" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 43611 } }
I20260806 22:53:26.017340 15036 leader_election.cc:288] T 38b53c157d264a9d84725854c91d8697 P 54b2b48fdddf4c6ea10ae001959e2ad3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 60837b341f5d4b3eb7e6a53780b4abd3 (127.28.218.195:38763), e9d32213de704457865ff3b95d803f34 (127.28.218.194:46483)
W20260806 22:53:26.017817 14619 leader_election.cc:341] T 38b53c157d264a9d84725854c91d8697 P 54b2b48fdddf4c6ea10ae001959e2ad3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 60837b341f5d4b3eb7e6a53780b4abd3 (127.28.218.195:38763): Not found: Tablet not found: 38b53c157d264a9d84725854c91d8697
W20260806 22:53:26.017885 14620 leader_election.cc:341] T 38b53c157d264a9d84725854c91d8697 P 54b2b48fdddf4c6ea10ae001959e2ad3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e9d32213de704457865ff3b95d803f34 (127.28.218.194:46483): Not found: Tablet not found: 38b53c157d264a9d84725854c91d8697
I20260806 22:53:26.017930 14619 leader_election.cc:302] T 38b53c157d264a9d84725854c91d8697 P 54b2b48fdddf4c6ea10ae001959e2ad3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54b2b48fdddf4c6ea10ae001959e2ad3; no voters: 60837b341f5d4b3eb7e6a53780b4abd3, e9d32213de704457865ff3b95d803f34
I20260806 22:53:26.018049 15036 raft_consensus.cc:2726] T 38b53c157d264a9d84725854c91d8697 P 54b2b48fdddf4c6ea10ae001959e2ad3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 22:53:26.282891 15037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:26.282634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:35241 (local address 127.28.218.193:33299)
0806 22:53:26.282782 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0806 22:53:26.282821 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:35241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":66,"threads_started":1}
I20260806 22:53:26.327477 15038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:26.327241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:35241 (local address 127.28.218.194:49931)
0806 22:53:26.327367 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0806 22:53:26.327417 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:35241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":75,"threads_started":1}
I20260806 22:53:26.448809 15039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:26.448601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:35241 (local address 127.28.218.195:47093)
0806 22:53:26.448727 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0806 22:53:26.448760 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:35241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":76,"threads_started":1}
I20260806 22:53:27.283629 15040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:27.283403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:35241 (local address 127.28.218.193:33385)
0806 22:53:27.283543 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0806 22:53:27.283581 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:35241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":59,"threads_started":1}
I20260806 22:53:27.328176 15041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:27.327986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:35241 (local address 127.28.218.194:56607)
0806 22:53:27.328101 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0806 22:53:27.328139 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:35241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":59,"threads_started":1}
I20260806 22:53:27.449395 15042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:27.449202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:35241 (local address 127.28.218.195:44297)
0806 22:53:27.449326 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0806 22:53:27.449360 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:35241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":65,"threads_started":1}
I20260806 22:53:28.284278 15043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:28.284082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:35241 (local address 127.28.218.193:42865)
0806 22:53:28.284199 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0806 22:53:28.284232 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:35241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":63,"threads_started":1}
I20260806 22:53:28.289011 15044 raft_consensus.cc:482] T 38b53c157d264a9d84725854c91d8697 P 54b2b48fdddf4c6ea10ae001959e2ad3 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:53:28.289100 15044 raft_consensus.cc:504] T 38b53c157d264a9d84725854c91d8697 P 54b2b48fdddf4c6ea10ae001959e2ad3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60837b341f5d4b3eb7e6a53780b4abd3" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 38763 } } peers { permanent_uuid: "e9d32213de704457865ff3b95d803f34" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 46483 } } peers { permanent_uuid: "54b2b48fdddf4c6ea10ae001959e2ad3" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 43611 } }
I20260806 22:53:28.289280 15044 leader_election.cc:288] T 38b53c157d264a9d84725854c91d8697 P 54b2b48fdddf4c6ea10ae001959e2ad3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 60837b341f5d4b3eb7e6a53780b4abd3 (127.28.218.195:38763), e9d32213de704457865ff3b95d803f34 (127.28.218.194:46483)
W20260806 22:53:28.289662 14620 leader_election.cc:341] T 38b53c157d264a9d84725854c91d8697 P 54b2b48fdddf4c6ea10ae001959e2ad3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e9d32213de704457865ff3b95d803f34 (127.28.218.194:46483): Not found: Tablet not found: 38b53c157d264a9d84725854c91d8697
W20260806 22:53:28.289745 14619 leader_election.cc:341] T 38b53c157d264a9d84725854c91d8697 P 54b2b48fdddf4c6ea10ae001959e2ad3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 60837b341f5d4b3eb7e6a53780b4abd3 (127.28.218.195:38763): Not found: Tablet not found: 38b53c157d264a9d84725854c91d8697
I20260806 22:53:28.289814 14619 leader_election.cc:302] T 38b53c157d264a9d84725854c91d8697 P 54b2b48fdddf4c6ea10ae001959e2ad3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54b2b48fdddf4c6ea10ae001959e2ad3; no voters: 60837b341f5d4b3eb7e6a53780b4abd3, e9d32213de704457865ff3b95d803f34
I20260806 22:53:28.289908 15044 raft_consensus.cc:2726] T 38b53c157d264a9d84725854c91d8697 P 54b2b48fdddf4c6ea10ae001959e2ad3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 22:53:28.328783 15045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:28.328567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:35241 (local address 127.28.218.194:41457)
0806 22:53:28.328696 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0806 22:53:28.328729 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:35241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":74,"threads_started":1}
I20260806 22:53:28.450168 15046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:28.449932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:35241 (local address 127.28.218.195:57265)
0806 22:53:28.450092 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0806 22:53:28.450127 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:35241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":74,"threads_started":1}
I20260806 22:53:29.285034 15048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:29.284799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:35241 (local address 127.28.218.193:58473)
0806 22:53:29.284950 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0806 22:53:29.284987 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:35241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":91,"threads_started":1}
I20260806 22:53:29.329348 15049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:29.329160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:35241 (local address 127.28.218.194:41599)
0806 22:53:29.329275 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0806 22:53:29.329309 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:35241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":61,"threads_started":1}
I20260806 22:53:29.450778 15050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:29.450582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:35241 (local address 127.28.218.195:38693)
0806 22:53:29.450703 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0806 22:53:29.450739 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:35241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":61,"threads_started":1}
I20260806 22:53:30.285773 15051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:30.285570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:35241 (local address 127.28.218.193:43771)
0806 22:53:30.285691 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0806 22:53:30.285726 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:35241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":62,"threads_started":1}
I20260806 22:53:30.330050 15052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:30.329806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:35241 (local address 127.28.218.194:58037)
0806 22:53:30.329959 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0806 22:53:30.329997 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:35241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":75,"threads_started":1}
I20260806 22:53:30.451438 15053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:30.451231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:35241 (local address 127.28.218.195:44235)
0806 22:53:30.451362 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0806 22:53:30.451398 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:35241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":60,"threads_started":1}
I20260806 22:53:31.286409 15054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:31.286203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:35241 (local address 127.28.218.193:46953)
0806 22:53:31.286328 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0806 22:53:31.286362 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:35241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":68,"threads_started":1}
I20260806 22:53:31.330710 15055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:31.330494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:35241 (local address 127.28.218.194:33503)
0806 22:53:31.330621 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0806 22:53:31.330658 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:35241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":63,"threads_started":1}
I20260806 22:53:31.452035 15056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:31.451837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:35241 (local address 127.28.218.195:37741)
0806 22:53:31.451955 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0806 22:53:31.451991 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:35241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":62,"threads_started":1}
I20260806 22:53:32.287050 15057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:32.286826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:35241 (local address 127.28.218.193:51989)
0806 22:53:32.286967 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0806 22:53:32.287003 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:35241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":84,"threads_started":1}
I20260806 22:53:32.331413 15058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:32.331201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:35241 (local address 127.28.218.194:50131)
0806 22:53:32.331332 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0806 22:53:32.331372 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:35241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":63,"threads_started":1}
I20260806 22:53:32.452654 15059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:32.452453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:35241 (local address 127.28.218.195:56457)
0806 22:53:32.452576 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0806 22:53:32.452611 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:35241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":62,"threads_started":1}
I20260806 22:53:33.287782 15060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:33.287554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:35241 (local address 127.28.218.193:55449)
0806 22:53:33.287688 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0806 22:53:33.287734 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:35241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":68,"threads_started":1}
I20260806 22:53:33.324200 15061 raft_consensus.cc:482] T 38b53c157d264a9d84725854c91d8697 P 54b2b48fdddf4c6ea10ae001959e2ad3 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:53:33.324285 15061 raft_consensus.cc:504] T 38b53c157d264a9d84725854c91d8697 P 54b2b48fdddf4c6ea10ae001959e2ad3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60837b341f5d4b3eb7e6a53780b4abd3" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 38763 } } peers { permanent_uuid: "e9d32213de704457865ff3b95d803f34" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 46483 } } peers { permanent_uuid: "54b2b48fdddf4c6ea10ae001959e2ad3" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 43611 } }
I20260806 22:53:33.324450 15061 leader_election.cc:288] T 38b53c157d264a9d84725854c91d8697 P 54b2b48fdddf4c6ea10ae001959e2ad3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 60837b341f5d4b3eb7e6a53780b4abd3 (127.28.218.195:38763), e9d32213de704457865ff3b95d803f34 (127.28.218.194:46483)
W20260806 22:53:33.324859 14619 leader_election.cc:341] T 38b53c157d264a9d84725854c91d8697 P 54b2b48fdddf4c6ea10ae001959e2ad3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 60837b341f5d4b3eb7e6a53780b4abd3 (127.28.218.195:38763): Not found: Tablet not found: 38b53c157d264a9d84725854c91d8697
W20260806 22:53:33.324951 14620 leader_election.cc:341] T 38b53c157d264a9d84725854c91d8697 P 54b2b48fdddf4c6ea10ae001959e2ad3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e9d32213de704457865ff3b95d803f34 (127.28.218.194:46483): Not found: Tablet not found: 38b53c157d264a9d84725854c91d8697
I20260806 22:53:33.325018 14620 leader_election.cc:302] T 38b53c157d264a9d84725854c91d8697 P 54b2b48fdddf4c6ea10ae001959e2ad3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54b2b48fdddf4c6ea10ae001959e2ad3; no voters: 60837b341f5d4b3eb7e6a53780b4abd3, e9d32213de704457865ff3b95d803f34
I20260806 22:53:33.325114 15061 raft_consensus.cc:2726] T 38b53c157d264a9d84725854c91d8697 P 54b2b48fdddf4c6ea10ae001959e2ad3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 22:53:33.332198 15062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:33.331986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:35241 (local address 127.28.218.194:34535)
0806 22:53:33.332120 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0806 22:53:33.332161 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:35241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":73,"threads_started":1}
I20260806 22:53:33.453507 15064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:33.453255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:35241 (local address 127.28.218.195:60049)
0806 22:53:33.453411 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0806 22:53:33.453461 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:35241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":67,"threads_started":1}
I20260806 22:53:34.288503 15065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:34.288265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:35241 (local address 127.28.218.193:56671)
0806 22:53:34.288407 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0806 22:53:34.288453 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:35241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":81,"threads_started":1}
I20260806 22:53:34.332811 15066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:34.332595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:35241 (local address 127.28.218.194:45555)
0806 22:53:34.332729 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0806 22:53:34.332766 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:35241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":64,"threads_started":1}
I20260806 22:53:34.454267 15067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:34.454041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:35241 (local address 127.28.218.195:33431)
0806 22:53:34.454182 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0806 22:53:34.454220 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:35241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":61,"threads_started":1}
I20260806 22:53:35.289170 15068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:35.288955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:35241 (local address 127.28.218.193:58623)
0806 22:53:35.289085 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0806 22:53:35.289120 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:35241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":64,"threads_started":1}
I20260806 22:53:35.333467 15069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:35.333234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:35241 (local address 127.28.218.194:40155)
0806 22:53:35.333373 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0806 22:53:35.333412 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:35241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":61,"threads_started":1}
I20260806 22:53:35.454991 15070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:35.454755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:35241 (local address 127.28.218.195:59607)
0806 22:53:35.454903 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0806 22:53:35.454939 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:35241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":68,"threads_started":1}
I20260806 22:53:36.290086 15072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:36.289758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:35241 (local address 127.28.218.193:41785)
0806 22:53:36.289977 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0806 22:53:36.290036 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:35241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":67,"threads_started":1}
I20260806 22:53:36.334240 15073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:36.334004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:35241 (local address 127.28.218.194:35265)
0806 22:53:36.334155 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0806 22:53:36.334194 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:35241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":89,"threads_started":1}
I20260806 22:53:36.455696 15074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:36.455466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:35241 (local address 127.28.218.195:34717)
0806 22:53:36.455619 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0806 22:53:36.455654 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:35241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":68,"threads_started":1}
I20260806 22:53:37.290866 15075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:37.290611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:35241 (local address 127.28.218.193:37749)
0806 22:53:37.290787 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0806 22:53:37.290822 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:35241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":110,"threads_started":1}
I20260806 22:53:37.334980 15076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:37.334740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:35241 (local address 127.28.218.194:50045)
0806 22:53:37.334889 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0806 22:53:37.334930 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:35241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":104,"threads_started":1}
I20260806 22:53:37.456338 15077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:37.456135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:35241 (local address 127.28.218.195:60847)
0806 22:53:37.456262 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0806 22:53:37.456296 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.254:35241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":65,"threads_started":1}
I20260806 22:53:37.463208 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 14566
I20260806 22:53:37.469841 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 14742
I20260806 22:53:37.474594 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 14885
2026-08-06T22:53:37Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58623 ms)
[----------] 30 tests from SecurityITest (230695 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056586788339-29547-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:53:37 dist-test-slave-1xrh krb5kdc[15080](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:53:37 dist-test-slave-1xrh krb5kdc[15080](info): set up 2 sockets
Aug 06 22:53:37 dist-test-slave-1xrh krb5kdc[15080](info): commencing operation
krb5kdc: starting...
W20260806 22:53:38.547061 29547 mini_kdc.cc:121] Time spent starting KDC: real 1.051s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056586788339-29547-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056586788339-29547-0/krb5kdc/test-user.keytab.
Aug 06 22:53:38 dist-test-slave-1xrh krb5kdc[15080](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056818, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:53:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:53:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.218.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.218.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.218.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab.
Entry for principal kudu/127.28.218.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab.
I20260806 22:53:38.670392 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056586788339-29547-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056586788339-29547-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056586788339-29547-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.218.254:37943
--webserver_interface=127.28.218.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab
--principal=kudu/127.28.218.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.218.212:35151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.218.254:37943
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:53:38.752388 15093 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:53:38.752633 15093 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:53:38.752672 15093 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:53:38.753789 15093 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:53:38.753861 15093 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:53:38.753885 15093 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:53:38.753906 15093 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:53:38.753930 15093 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:53:38.755440 15093 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:35151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056586788339-29547-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056586788339-29547-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.218.254:37943
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab
--principal=kudu/127.28.218.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.218.254:37943
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.218.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15093
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056586788339-29547-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:53:38.755718 15093 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:53:38.755939 15093 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:53:38.761029 15098 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:53:38.761165 15093 server_base.cc:1034] running on GCE node
W20260806 22:53:38.761070 15099 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:53:38.761044 15101 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:53:38.761466 15093 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:53:38.761767 15093 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:53:38.762967 15093 hybrid_clock.cc:648] HybridClock initialized: now 1786056818762950 us; error 50 us; skew 500 ppm
Aug 06 22:53:38 dist-test-slave-1xrh krb5kdc[15080](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056818, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:53:38.765481 15093 init.cc:377] Logged in from keytab as kudu/127.28.218.254@KRBTEST.COM (short username kudu)
I20260806 22:53:38.765961 15093 webserver.cc:460] Webserver started at http://127.28.218.254:34521/ using document root <none> and password file <none>
I20260806 22:53:38.766191 15093 fs_manager.cc:359] Metadata directory not provided
I20260806 22:53:38.766259 15093 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:53:38.766407 15093 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:53:38.767513 15093 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056586788339-29547-0/minicluster-data/master-0/data/instance:
uuid: "3ad431b9c0cf44c383918410d4f3e7c3"
format_stamp: "Formatted at 2026-08-06 22:53:38 on dist-test-slave-1xrh"
server_key: "44f92f11fb19cff429cbb7db02363f1b"
server_key_iv: "b82d0a77c12e1cbda4d98b33a53c9ef4"
server_key_version: "encryptionkey@0"
I20260806 22:53:38.767894 15093 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056586788339-29547-0/minicluster-data/master-0/wal/instance:
uuid: "3ad431b9c0cf44c383918410d4f3e7c3"
format_stamp: "Formatted at 2026-08-06 22:53:38 on dist-test-slave-1xrh"
server_key: "44f92f11fb19cff429cbb7db02363f1b"
server_key_iv: "b82d0a77c12e1cbda4d98b33a53c9ef4"
server_key_version: "encryptionkey@0"
I20260806 22:53:38.769232 15093 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:53:38.769999 15108 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:53:38.770179 15093 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:53:38.770280 15093 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056586788339-29547-0/minicluster-data/master-0/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056586788339-29547-0/minicluster-data/master-0/wal
uuid: "3ad431b9c0cf44c383918410d4f3e7c3"
format_stamp: "Formatted at 2026-08-06 22:53:38 on dist-test-slave-1xrh"
server_key: "44f92f11fb19cff429cbb7db02363f1b"
server_key_iv: "b82d0a77c12e1cbda4d98b33a53c9ef4"
server_key_version: "encryptionkey@0"
I20260806 22:53:38.770375 15093 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056586788339-29547-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056586788339-29547-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056586788339-29547-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:53:38.781193 15093 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:53:38.782982 15093 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:53:38.783202 15093 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:53:38.787492 15093 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.254:37943
I20260806 22:53:38.787535 15163 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.254:37943 every 8 connection(s)
I20260806 22:53:38.787894 15093 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
I20260806 22:53:38.788724 15164 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:53:38.790940 15164 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ad431b9c0cf44c383918410d4f3e7c3: Bootstrap starting.
I20260806 22:53:38.791597 15164 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3ad431b9c0cf44c383918410d4f3e7c3: Neither blocks nor log segments found. Creating new log.
I20260806 22:53:38.791903 15164 log.cc:824] T 00000000000000000000000000000000 P 3ad431b9c0cf44c383918410d4f3e7c3: Log is configured to *not* fsync() on all Append() calls
I20260806 22:53:38.792611 15164 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ad431b9c0cf44c383918410d4f3e7c3: No bootstrap required, opened a new log
I20260806 22:53:38.794238 15164 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3ad431b9c0cf44c383918410d4f3e7c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ad431b9c0cf44c383918410d4f3e7c3" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 37943 } }
I20260806 22:53:38.794370 15164 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3ad431b9c0cf44c383918410d4f3e7c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:53:38.794391 15164 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3ad431b9c0cf44c383918410d4f3e7c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ad431b9c0cf44c383918410d4f3e7c3, State: Initialized, Role: FOLLOWER
I20260806 22:53:38.794490 15164 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3ad431b9c0cf44c383918410d4f3e7c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ad431b9c0cf44c383918410d4f3e7c3" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 37943 } }
I20260806 22:53:38.794579 15164 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3ad431b9c0cf44c383918410d4f3e7c3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:53:38.794608 15164 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3ad431b9c0cf44c383918410d4f3e7c3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:53:38.794653 15164 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3ad431b9c0cf44c383918410d4f3e7c3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:53:38.795249 15164 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3ad431b9c0cf44c383918410d4f3e7c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ad431b9c0cf44c383918410d4f3e7c3" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 37943 } }
I20260806 22:53:38.795260 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 15093
I20260806 22:53:38.795384 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056586788339-29547-0/minicluster-data/master-0/wal/instance
I20260806 22:53:38.795370 15164 leader_election.cc:302] T 00000000000000000000000000000000 P 3ad431b9c0cf44c383918410d4f3e7c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3ad431b9c0cf44c383918410d4f3e7c3; no voters:
I20260806 22:53:38.795555 15164 leader_election.cc:288] T 00000000000000000000000000000000 P 3ad431b9c0cf44c383918410d4f3e7c3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:53:38.795606 15167 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3ad431b9c0cf44c383918410d4f3e7c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:53:38.795655 29547 external_mini_cluster.cc:1428] Setting key 6ed3053bd133e5de03e19df1281c1531
I20260806 22:53:38.795810 15167 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3ad431b9c0cf44c383918410d4f3e7c3 [term 1 LEADER]: Becoming Leader. State: Replica: 3ad431b9c0cf44c383918410d4f3e7c3, State: Running, Role: LEADER
I20260806 22:53:38.795862 15164 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3ad431b9c0cf44c383918410d4f3e7c3 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:53:38.795897 15167 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3ad431b9c0cf44c383918410d4f3e7c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ad431b9c0cf44c383918410d4f3e7c3" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 37943 } }
I20260806 22:53:38.796213 15167 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ad431b9c0cf44c383918410d4f3e7c3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3ad431b9c0cf44c383918410d4f3e7c3. Latest consensus state: current_term: 1 leader_uuid: "3ad431b9c0cf44c383918410d4f3e7c3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ad431b9c0cf44c383918410d4f3e7c3" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 37943 } } }
I20260806 22:53:38.796341 15167 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ad431b9c0cf44c383918410d4f3e7c3 [sys.catalog]: This master's current role is: LEADER
I20260806 22:53:38.796442 15168 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ad431b9c0cf44c383918410d4f3e7c3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3ad431b9c0cf44c383918410d4f3e7c3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ad431b9c0cf44c383918410d4f3e7c3" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 37943 } } }
I20260806 22:53:38.796583 15168 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ad431b9c0cf44c383918410d4f3e7c3 [sys.catalog]: This master's current role is: LEADER
I20260806 22:53:38.796725 15177 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:53:38.797329 15177 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:53:38.798902 15177 catalog_manager.cc:1347] Generated new cluster ID: dbb93559d28c415f982635830d16f822
I20260806 22:53:38.798957 15177 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 22:53:38 dist-test-slave-1xrh krb5kdc[15080](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056818, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.218.254@KRBTEST.COM
I20260806 22:53:38.804874 15177 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:53:38.805538 15177 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:53:38.806982 15172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:38.796141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55962 (local address 127.28.218.254:37943)
0806 22:53:38.796739 (+ 598us) server_negotiation.cc:206] Beginning negotiation
0806 22:53:38.796748 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:53:38.796793 (+ 45us) server_negotiation.cc:407] Connection header received
0806 22:53:38.796935 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:53:38.796942 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:53:38.797006 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:53:38.797869 (+ 863us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:53:38.798067 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:38.799231 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:38.799984 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:38.800400 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:38.800534 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:38.802955 (+ 2421us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:53:38.802958 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:53:38.802959 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:53:38.802971 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:53:38.805452 (+ 2481us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:53:38.805851 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:53:38.805854 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:53:38.805854 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:53:38.805943 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:53:38.806185 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:53:38.806186 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:53:38.806187 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:53:38.806408 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:53:38.806600 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0806 22:53:38.806666 (+ 66us) server_negotiation.cc:299] Negotiation successful
0806 22:53:38.806814 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":546,"thread_start_us":62,"threads_started":1}
I20260806 22:53:38.817137 15177 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3ad431b9c0cf44c383918410d4f3e7c3: Generated new TSK 0
I20260806 22:53:38.817317 15177 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.28.218.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.218.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.218.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.193.keytab.
Entry for principal kudu/127.28.218.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.193.keytab.
I20260806 22:53:38.851436 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056586788339-29547-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056586788339-29547-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056586788339-29547-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056586788339-29547-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.218.193:0
--local_ip_for_outbound_sockets=127.28.218.193
--webserver_interface=127.28.218.193
--webserver_port=0
--tserver_master_addrs=127.28.218.254:37943
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.193.keytab
--principal=kudu/127.28.218.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.218.212:35151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:53:38.942394 15190 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:53:38.942608 15190 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:53:38.942631 15190 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:53:38.943779 15190 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:53:38.943864 15190 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:53:38.943892 15190 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.218.193
I20260806 22:53:38.945358 15190 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:35151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056586788339-29547-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056586788339-29547-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.193.keytab
--principal=kudu/127.28.218.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.218.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056586788339-29547-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.218.193
--webserver_port=0
--tserver_master_addrs=127.28.218.254:37943
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15190
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.218.193
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056586788339-29547-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:53:38.945608 15190 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:53:38.945860 15190 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:53:38.951920 15198 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:53:38.951908 15197 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:53:38.952016 15200 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:53:38.952214 15190 server_base.cc:1034] running on GCE node
I20260806 22:53:38.952397 15190 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:53:38.952657 15190 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:53:38.953856 15190 hybrid_clock.cc:648] HybridClock initialized: now 1786056818953856 us; error 43 us; skew 500 ppm
Aug 06 22:53:38 dist-test-slave-1xrh krb5kdc[15080](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056818, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:53:38.956477 15190 init.cc:377] Logged in from keytab as kudu/127.28.218.193@KRBTEST.COM (short username kudu)
I20260806 22:53:38.956974 15190 webserver.cc:460] Webserver started at http://127.28.218.193:33349/ using document root <none> and password file <none>
I20260806 22:53:38.957271 15190 fs_manager.cc:359] Metadata directory not provided
I20260806 22:53:38.957326 15190 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:53:38.957417 15190 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:53:38.958412 15190 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056586788339-29547-0/minicluster-data/ts-0/data/instance:
uuid: "67bb7fd16e8b4d60b9e82d20acffab4e"
format_stamp: "Formatted at 2026-08-06 22:53:38 on dist-test-slave-1xrh"
server_key: "ea5a49679f824c1f8c5b3eb8b0fd913b"
server_key_iv: "08f9dbb8c0a6b71cd634abda2ef012e8"
server_key_version: "encryptionkey@0"
I20260806 22:53:38.958720 15190 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056586788339-29547-0/minicluster-data/ts-0/wal/instance:
uuid: "67bb7fd16e8b4d60b9e82d20acffab4e"
format_stamp: "Formatted at 2026-08-06 22:53:38 on dist-test-slave-1xrh"
server_key: "ea5a49679f824c1f8c5b3eb8b0fd913b"
server_key_iv: "08f9dbb8c0a6b71cd634abda2ef012e8"
server_key_version: "encryptionkey@0"
I20260806 22:53:38.960098 15190 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:53:38.960809 15207 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:53:38.961017 15190 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:53:38.961112 15190 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056586788339-29547-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056586788339-29547-0/minicluster-data/ts-0/wal
uuid: "67bb7fd16e8b4d60b9e82d20acffab4e"
format_stamp: "Formatted at 2026-08-06 22:53:38 on dist-test-slave-1xrh"
server_key: "ea5a49679f824c1f8c5b3eb8b0fd913b"
server_key_iv: "08f9dbb8c0a6b71cd634abda2ef012e8"
server_key_version: "encryptionkey@0"
I20260806 22:53:38.961186 15190 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056586788339-29547-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056586788339-29547-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056586788339-29547-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:53:38.978370 15190 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:53:38.980798 15190 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:53:38.980971 15190 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:53:38.981221 15190 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:53:38.981549 15190 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:53:38.981606 15190 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:53:38.981644 15190 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:53:38.981671 15190 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:53:38.987751 15190 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.193:40633
I20260806 22:53:38.987802 15326 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.193:40633 every 8 connection(s)
I20260806 22:53:38.988106 15190 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056586788339-29547-0/minicluster-data/ts-0/data/info.pb
Aug 06 22:53:38 dist-test-slave-1xrh krb5kdc[15080](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056818, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.193@KRBTEST.COM for kudu/127.28.218.254@KRBTEST.COM
I20260806 22:53:38.997238 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 15190
I20260806 22:53:38.997363 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056586788339-29547-0/minicluster-data/ts-0/wal/instance
I20260806 22:53:38.997646 29547 external_mini_cluster.cc:1428] Setting key c070634db5a86635a67114929ad7bb11
I20260806 22:53:38.998086 15172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:38.988849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.193:33987 (local address 127.28.218.254:37943)
0806 22:53:38.988893 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 22:53:38.988895 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:53:38.989163 (+ 268us) server_negotiation.cc:407] Connection header received
0806 22:53:38.989694 (+ 531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:53:38.989695 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:53:38.989709 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:53:38.989805 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:53:38.990769 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:38.991356 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:38.992377 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:38.992584 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:38.992664 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:38.994888 (+ 2224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:53:38.994890 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:53:38.994890 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:53:38.994898 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:53:38.996386 (+ 1488us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:53:38.996825 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:53:38.996826 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:53:38.996827 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:53:38.996894 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:53:38.997367 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:53:38.997368 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:53:38.997368 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:53:38.997512 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:53:38.997574 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 22:53:38.997905 (+ 331us) server_negotiation.cc:299] Negotiation successful
0806 22:53:38.997994 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 22:53:38.998279 15329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:38.988834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:37943 (local address 127.28.218.193:33987)
0806 22:53:38.989035 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0806 22:53:38.989070 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 22:53:38.989579 (+ 509us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:53:38.989883 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:53:38.989888 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:53:38.990447 (+ 559us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:53:38.990638 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:53:38.990643 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:53:38.991456 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:53:38.991457 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:53:38.992254 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:53:38.992257 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:53:38.992685 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:53:38.992687 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:53:38.992845 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:38.993393 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:53:38.993401 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:53:38.994780 (+ 1379us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:53:38.996476 (+ 1696us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:53:38.996483 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:53:38.996485 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:53:38.996725 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:53:38.996969 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:53:38.996970 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:53:38.996970 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:53:38.997197 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:53:38.997588 (+ 391us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:53:38.997595 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:53:38.997761 (+ 166us) client_negotiation.cc:769] Sending connection context
0806 22:53:38.997880 (+ 119us) client_negotiation.cc:240] Negotiation successful
0806 22:53:38.998049 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":89,"threads_started":1}
I20260806 22:53:38.998845 15327 heartbeater.cc:347] Connected to a master server at 127.28.218.254:37943
I20260806 22:53:38.998939 15327 heartbeater.cc:464] Registering TS with master...
I20260806 22:53:38.999191 15327 heartbeater.cc:511] Master 127.28.218.254:37943 requested a full tablet report, sending...
I20260806 22:53:38.999730 15127 ts_manager.cc:194] Registered new tserver with Master: 67bb7fd16e8b4d60b9e82d20acffab4e (127.28.218.193:40633)
I20260806 22:53:39.000339 15127 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.218.193@KRBTEST.COM'} at 127.28.218.193:33987
WARNING: no policy specified for kudu/127.28.218.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.218.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.218.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.194.keytab.
Entry for principal kudu/127.28.218.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.194.keytab.
I20260806 22:53:39.026247 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056586788339-29547-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056586788339-29547-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056586788339-29547-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056586788339-29547-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.218.194:0
--local_ip_for_outbound_sockets=127.28.218.194
--webserver_interface=127.28.218.194
--webserver_port=0
--tserver_master_addrs=127.28.218.254:37943
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.194.keytab
--principal=kudu/127.28.218.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.218.212:35151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:53:39.106678 15332 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:53:39.106854 15332 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:53:39.106875 15332 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:53:39.107920 15332 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:53:39.107954 15332 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:53:39.107983 15332 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.218.194
I20260806 22:53:39.109194 15332 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:35151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056586788339-29547-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056586788339-29547-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.194.keytab
--principal=kudu/127.28.218.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.218.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056586788339-29547-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.218.194
--webserver_port=0
--tserver_master_addrs=127.28.218.254:37943
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15332
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.218.194
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056586788339-29547-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:53:39.109357 15332 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:53:39.109532 15332 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:53:39.114454 15339 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:53:39.114471 15338 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:53:39.114552 15332 server_base.cc:1034] running on GCE node
W20260806 22:53:39.114480 15342 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:53:39.114862 15332 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:53:39.115172 15332 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:53:39.116366 15332 hybrid_clock.cc:648] HybridClock initialized: now 1786056819116355 us; error 24 us; skew 500 ppm
Aug 06 22:53:39 dist-test-slave-1xrh krb5kdc[15080](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056819, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:53:39.118687 15332 init.cc:377] Logged in from keytab as kudu/127.28.218.194@KRBTEST.COM (short username kudu)
I20260806 22:53:39.119199 15332 webserver.cc:460] Webserver started at http://127.28.218.194:44219/ using document root <none> and password file <none>
I20260806 22:53:39.119439 15332 fs_manager.cc:359] Metadata directory not provided
I20260806 22:53:39.119504 15332 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:53:39.119652 15332 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:53:39.120574 15332 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056586788339-29547-0/minicluster-data/ts-1/data/instance:
uuid: "18509d307ad44c9ba4d309e2c15da3bb"
format_stamp: "Formatted at 2026-08-06 22:53:39 on dist-test-slave-1xrh"
server_key: "97f006df633afcc816d36bb68f75990e"
server_key_iv: "a896f34c71fd4fb2329b263dc9ed1649"
server_key_version: "encryptionkey@0"
I20260806 22:53:39.121011 15332 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056586788339-29547-0/minicluster-data/ts-1/wal/instance:
uuid: "18509d307ad44c9ba4d309e2c15da3bb"
format_stamp: "Formatted at 2026-08-06 22:53:39 on dist-test-slave-1xrh"
server_key: "97f006df633afcc816d36bb68f75990e"
server_key_iv: "a896f34c71fd4fb2329b263dc9ed1649"
server_key_version: "encryptionkey@0"
I20260806 22:53:39.122315 15332 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:53:39.122972 15349 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:53:39.123154 15332 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:53:39.123258 15332 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056586788339-29547-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056586788339-29547-0/minicluster-data/ts-1/wal
uuid: "18509d307ad44c9ba4d309e2c15da3bb"
format_stamp: "Formatted at 2026-08-06 22:53:39 on dist-test-slave-1xrh"
server_key: "97f006df633afcc816d36bb68f75990e"
server_key_iv: "a896f34c71fd4fb2329b263dc9ed1649"
server_key_version: "encryptionkey@0"
I20260806 22:53:39.123342 15332 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056586788339-29547-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056586788339-29547-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056586788339-29547-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:53:39.141089 15332 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:53:39.143056 15332 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:53:39.143236 15332 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:53:39.143462 15332 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:53:39.143757 15332 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:53:39.143841 15332 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:53:39.143904 15332 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:53:39.143926 15332 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:53:39.150265 15332 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.194:42119
I20260806 22:53:39.150305 15471 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.194:42119 every 8 connection(s)
I20260806 22:53:39.150635 15332 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056586788339-29547-0/minicluster-data/ts-1/data/info.pb
I20260806 22:53:39.151232 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 15332
I20260806 22:53:39.151364 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056586788339-29547-0/minicluster-data/ts-1/wal/instance
I20260806 22:53:39.151600 29547 external_mini_cluster.cc:1428] Setting key bdda2cf54910d6e23cf9419ca55fb324
Aug 06 22:53:39 dist-test-slave-1xrh krb5kdc[15080](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056819, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.194@KRBTEST.COM for kudu/127.28.218.254@KRBTEST.COM
I20260806 22:53:39.161563 15172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:39.151737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.194:44843 (local address 127.28.218.254:37943)
0806 22:53:39.151895 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0806 22:53:39.151897 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:53:39.151909 (+ 12us) server_negotiation.cc:407] Connection header received
0806 22:53:39.152448 (+ 539us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:53:39.152449 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:53:39.152462 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:53:39.152543 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:53:39.153437 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:39.154199 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:39.155703 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:39.156030 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:39.156106 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:39.158285 (+ 2179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:53:39.158287 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:53:39.158287 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:53:39.158296 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:53:39.160054 (+ 1758us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:53:39.160461 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:53:39.160463 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:53:39.160464 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:53:39.160533 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:53:39.160861 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:53:39.160863 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:53:39.160863 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:53:39.161011 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:53:39.161096 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 22:53:39.161364 (+ 268us) server_negotiation.cc:299] Negotiation successful
0806 22:53:39.161451 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130}
WARNING: no policy specified for kudu/127.28.218.195@KRBTEST.COM; defaulting to no policy
I20260806 22:53:39.161593 15474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:39.151434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:37943 (local address 127.28.218.194:44843)
0806 22:53:39.151615 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0806 22:53:39.151675 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0806 22:53:39.152331 (+ 656us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:53:39.152594 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:53:39.152598 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:53:39.153147 (+ 549us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:53:39.153328 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:53:39.153332 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:53:39.154320 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:53:39.154322 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:53:39.155608 (+ 1286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:53:39.155611 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:53:39.156116 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:53:39.156116 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:53:39.156215 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:39.156763 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:53:39.156777 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:53:39.158189 (+ 1412us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:53:39.160142 (+ 1953us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:53:39.160148 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:53:39.160149 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:53:39.160372 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:53:39.160655 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:53:39.160656 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:53:39.160657 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:53:39.160773 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:53:39.161097 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:53:39.161102 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:53:39.161219 (+ 117us) client_negotiation.cc:769] Sending connection context
0806 22:53:39.161308 (+ 89us) client_negotiation.cc:240] Negotiation successful
0806 22:53:39.161429 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":99,"threads_started":1}
I20260806 22:53:39.162211 15472 heartbeater.cc:347] Connected to a master server at 127.28.218.254:37943
I20260806 22:53:39.162293 15472 heartbeater.cc:464] Registering TS with master...
I20260806 22:53:39.162497 15472 heartbeater.cc:511] Master 127.28.218.254:37943 requested a full tablet report, sending...
I20260806 22:53:39.162999 15127 ts_manager.cc:194] Registered new tserver with Master: 18509d307ad44c9ba4d309e2c15da3bb (127.28.218.194:42119)
I20260806 22:53:39.163408 15127 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.218.194@KRBTEST.COM'} at 127.28.218.194:44843
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.218.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.218.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.195.keytab.
Entry for principal kudu/127.28.218.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.195.keytab.
I20260806 22:53:39.180205 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056586788339-29547-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056586788339-29547-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056586788339-29547-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056586788339-29547-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.218.195:0
--local_ip_for_outbound_sockets=127.28.218.195
--webserver_interface=127.28.218.195
--webserver_port=0
--tserver_master_addrs=127.28.218.254:37943
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.195.keytab
--principal=kudu/127.28.218.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.218.212:35151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:53:39.259063 15477 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:53:39.259275 15477 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:53:39.259299 15477 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:53:39.260406 15477 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:53:39.260444 15477 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:53:39.260471 15477 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.218.195
I20260806 22:53:39.261821 15477 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:35151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056586788339-29547-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056586788339-29547-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.195.keytab
--principal=kudu/127.28.218.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.218.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056586788339-29547-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.218.195
--webserver_port=0
--tserver_master_addrs=127.28.218.254:37943
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15477
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.218.195
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056586788339-29547-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:53:39.261998 15477 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:53:39.262177 15477 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:53:39.267217 15482 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:53:39.267205 15483 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:53:39.267405 15477 server_base.cc:1034] running on GCE node
W20260806 22:53:39.267205 15485 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:53:39.267704 15477 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:53:39.267899 15477 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:53:39.269042 15477 hybrid_clock.cc:648] HybridClock initialized: now 1786056819269028 us; error 25 us; skew 500 ppm
Aug 06 22:53:39 dist-test-slave-1xrh krb5kdc[15080](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056819, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:53:39.271423 15477 init.cc:377] Logged in from keytab as kudu/127.28.218.195@KRBTEST.COM (short username kudu)
I20260806 22:53:39.271795 15477 webserver.cc:460] Webserver started at http://127.28.218.195:45357/ using document root <none> and password file <none>
I20260806 22:53:39.271966 15477 fs_manager.cc:359] Metadata directory not provided
I20260806 22:53:39.272009 15477 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:53:39.272099 15477 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:53:39.273113 15477 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056586788339-29547-0/minicluster-data/ts-2/data/instance:
uuid: "b1ee65c704e642499e51462281b5d15c"
format_stamp: "Formatted at 2026-08-06 22:53:39 on dist-test-slave-1xrh"
server_key: "e309fa67f2646f35e798ebb99a42170a"
server_key_iv: "28434498bfc36d26c8cbd41ef5e768f7"
server_key_version: "encryptionkey@0"
I20260806 22:53:39.273413 15477 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056586788339-29547-0/minicluster-data/ts-2/wal/instance:
uuid: "b1ee65c704e642499e51462281b5d15c"
format_stamp: "Formatted at 2026-08-06 22:53:39 on dist-test-slave-1xrh"
server_key: "e309fa67f2646f35e798ebb99a42170a"
server_key_iv: "28434498bfc36d26c8cbd41ef5e768f7"
server_key_version: "encryptionkey@0"
I20260806 22:53:39.274785 15477 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 22:53:39.275487 15493 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:53:39.275669 15477 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 22:53:39.275748 15477 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056586788339-29547-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056586788339-29547-0/minicluster-data/ts-2/wal
uuid: "b1ee65c704e642499e51462281b5d15c"
format_stamp: "Formatted at 2026-08-06 22:53:39 on dist-test-slave-1xrh"
server_key: "e309fa67f2646f35e798ebb99a42170a"
server_key_iv: "28434498bfc36d26c8cbd41ef5e768f7"
server_key_version: "encryptionkey@0"
I20260806 22:53:39.275800 15477 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056586788339-29547-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056586788339-29547-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056586788339-29547-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:53:39.289816 15477 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:53:39.291617 15477 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:53:39.291725 15477 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:53:39.291896 15477 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:53:39.292213 15477 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:53:39.292246 15477 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:53:39.292266 15477 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:53:39.292279 15477 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:53:39.298602 15477 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.195:46717
I20260806 22:53:39.298681 15614 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.195:46717 every 8 connection(s)
I20260806 22:53:39.298985 15477 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056586788339-29547-0/minicluster-data/ts-2/data/info.pb
Aug 06 22:53:39 dist-test-slave-1xrh krb5kdc[15080](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056819, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.195@KRBTEST.COM for kudu/127.28.218.254@KRBTEST.COM
I20260806 22:53:39.304934 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 15477
I20260806 22:53:39.305074 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056586788339-29547-0/minicluster-data/ts-2/wal/instance
I20260806 22:53:39.305297 29547 external_mini_cluster.cc:1428] Setting key c923d04dd84e451fcdb2c193b0683d20
I20260806 22:53:39.308250 15172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:39.299692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.195:48033 (local address 127.28.218.254:37943)
0806 22:53:39.299736 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 22:53:39.299738 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:53:39.299941 (+ 203us) server_negotiation.cc:407] Connection header received
0806 22:53:39.300362 (+ 421us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:53:39.300363 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:53:39.300375 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:53:39.300425 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:53:39.301318 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:39.301811 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:39.302740 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:39.302928 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:39.302975 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:39.305166 (+ 2191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:53:39.305168 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:53:39.305168 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:53:39.305176 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:53:39.306912 (+ 1736us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:53:39.307453 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:53:39.307454 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:53:39.307455 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:53:39.307508 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:53:39.307698 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:53:39.307699 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:53:39.307699 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:53:39.307798 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:53:39.307860 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 22:53:39.308126 (+ 266us) server_negotiation.cc:299] Negotiation successful
0806 22:53:39.308190 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 22:53:39.308421 15617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:39.299661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:37943 (local address 127.28.218.195:48033)
0806 22:53:39.299820 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0806 22:53:39.299854 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0806 22:53:39.300285 (+ 431us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:53:39.300494 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:53:39.300498 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:53:39.301060 (+ 562us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:53:39.301245 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:53:39.301249 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:53:39.301892 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:53:39.301893 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:53:39.302666 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:53:39.302667 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:53:39.303001 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:53:39.303002 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:53:39.303107 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:39.303556 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:53:39.303563 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:53:39.305049 (+ 1486us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:53:39.307012 (+ 1963us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:53:39.307019 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:53:39.307021 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:53:39.307381 (+ 360us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:53:39.307555 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:53:39.307556 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:53:39.307556 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:53:39.307643 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:53:39.307897 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:53:39.307902 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:53:39.308012 (+ 110us) client_negotiation.cc:769] Sending connection context
0806 22:53:39.308094 (+ 82us) client_negotiation.cc:240] Negotiation successful
0806 22:53:39.308209 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":98,"threads_started":1}
I20260806 22:53:39.309001 15615 heartbeater.cc:347] Connected to a master server at 127.28.218.254:37943
I20260806 22:53:39.309123 15615 heartbeater.cc:464] Registering TS with master...
I20260806 22:53:39.309304 15615 heartbeater.cc:511] Master 127.28.218.254:37943 requested a full tablet report, sending...
I20260806 22:53:39.309736 15127 ts_manager.cc:194] Registered new tserver with Master: b1ee65c704e642499e51462281b5d15c (127.28.218.195:46717)
I20260806 22:53:39.310133 15127 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.218.195@KRBTEST.COM'} at 127.28.218.195:48033
I20260806 22:53:39.317332 29547 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:53:39.325174 15172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:39.318790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55974 (local address 127.28.218.254:37943)
0806 22:53:39.318874 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0806 22:53:39.318876 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:53:39.318985 (+ 109us) server_negotiation.cc:407] Connection header received
0806 22:53:39.319060 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:53:39.319061 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:53:39.319072 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:53:39.319161 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:53:39.319741 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:39.320232 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:39.320991 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:39.321205 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:39.321298 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:39.322597 (+ 1299us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:53:39.322598 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:53:39.322598 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:53:39.322605 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:53:39.323935 (+ 1330us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:53:39.324311 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:53:39.324313 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:53:39.324313 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:53:39.324376 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:53:39.324606 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:53:39.324607 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:53:39.324607 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:53:39.324768 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:53:39.324878 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 22:53:39.324992 (+ 114us) server_negotiation.cc:299] Negotiation successful
0806 22:53:39.325072 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 22:53:39.326859 15127 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55974:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:53:39.328646 15127 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:53:39.335733 15631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:39.331360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.194:42119 (local address 127.0.0.1:43862)
0806 22:53:39.331885 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0806 22:53:39.331896 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:53:39.331958 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:53:39.332257 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:53:39.332261 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:53:39.332264 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:53:39.332332 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:53:39.332334 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:53:39.333295 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:53:39.333296 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:53:39.334466 (+ 1170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:53:39.334468 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:53:39.335400 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:53:39.335401 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:53:39.335490 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:39.335504 (+ 14us) client_negotiation.cc:769] Sending connection context
0806 22:53:39.335600 (+ 96us) client_negotiation.cc:240] Negotiation successful
0806 22:53:39.335658 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":485,"mutex_wait_us":2,"thread_start_us":81,"threads_started":1}
I20260806 22:53:39.335747 15630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:39.331469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43862 (local address 127.28.218.194:42119)
0806 22:53:39.331709 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0806 22:53:39.331714 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:53:39.332057 (+ 343us) server_negotiation.cc:407] Connection header received
0806 22:53:39.332108 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:53:39.332111 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:53:39.332129 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:53:39.332243 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:53:39.332418 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:39.333183 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:39.334564 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:39.335311 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:39.335403 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:39.335464 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 22:53:39.335618 (+ 154us) server_negotiation.cc:299] Negotiation successful
0806 22:53:39.335659 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":196,"thread_start_us":124,"threads_started":1}
I20260806 22:53:39.336167 15628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:39.331251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60122 (local address 127.28.218.193:40633)
0806 22:53:39.332232 (+ 981us) server_negotiation.cc:206] Beginning negotiation
0806 22:53:39.332237 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:53:39.332246 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:53:39.332285 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:53:39.332287 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:53:39.332301 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:53:39.332379 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:53:39.332497 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:39.333638 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:39.334968 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:39.335702 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:39.335947 (+ 245us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:39.336013 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 22:53:39.336037 (+ 24us) server_negotiation.cc:299] Negotiation successful
0806 22:53:39.336074 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":944,"thread_start_us":77,"threads_started":1}
I20260806 22:53:39.336222 15627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:39.331106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.193:40633 (local address 127.0.0.1:60122)
0806 22:53:39.331263 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0806 22:53:39.331304 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 22:53:39.331368 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:53:39.332376 (+ 1008us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:53:39.332376 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:53:39.332379 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:53:39.332426 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:53:39.332428 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:53:39.333750 (+ 1322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:53:39.333752 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:53:39.334875 (+ 1123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:53:39.334877 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:53:39.335793 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:53:39.335794 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:53:39.335846 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:39.335849 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:53:39.335876 (+ 27us) client_negotiation.cc:240] Negotiation successful
0806 22:53:39.335908 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"mutex_wait_us":39,"thread_start_us":96,"threads_started":1}
I20260806 22:53:39.336668 15400 tablet_service.cc:1467] Processing CreateTablet for tablet 1d8ba8971e82402c95e02ca10dee1fb6 (DEFAULT_TABLE table=test-table [id=63a27bbb37cb42e3b4c6bc7f5f86584d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:53:39.337038 15400 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1d8ba8971e82402c95e02ca10dee1fb6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:53:39.339579 15629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:39.331546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.195:46717 (local address 127.0.0.1:33602)
0806 22:53:39.332035 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0806 22:53:39.332043 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 22:53:39.332102 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:53:39.332353 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:53:39.332355 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:53:39.332357 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:53:39.332407 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:53:39.332409 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:53:39.333766 (+ 1357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:53:39.333767 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:53:39.334677 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:53:39.334680 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:53:39.335642 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:53:39.335643 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:53:39.335706 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:39.335711 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:53:39.335742 (+ 31us) client_negotiation.cc:240] Negotiation successful
0806 22:53:39.335792 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":470,"mutex_wait_us":1,"thread_start_us":80,"threads_started":1}
I20260806 22:53:39.339818 15633 tablet_bootstrap.cc:492] T 1d8ba8971e82402c95e02ca10dee1fb6 P 18509d307ad44c9ba4d309e2c15da3bb: Bootstrap starting.
I20260806 22:53:39.339890 15632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:39.331700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33602 (local address 127.28.218.195:46717)
0806 22:53:39.331886 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0806 22:53:39.331891 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:53:39.332104 (+ 213us) server_negotiation.cc:407] Connection header received
0806 22:53:39.332176 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:53:39.332178 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:53:39.332196 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:53:39.332274 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:53:39.332489 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:39.333626 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:39.334789 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:39.335534 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:39.339579 (+ 4045us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:39.339644 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 22:53:39.339670 (+ 26us) server_negotiation.cc:299] Negotiation successful
0806 22:53:39.339715 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":87,"threads_started":1}
I20260806 22:53:39.339910 15257 tablet_service.cc:1467] Processing CreateTablet for tablet 1d8ba8971e82402c95e02ca10dee1fb6 (DEFAULT_TABLE table=test-table [id=63a27bbb37cb42e3b4c6bc7f5f86584d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:53:39.340313 15257 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1d8ba8971e82402c95e02ca10dee1fb6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:53:39.340401 15547 tablet_service.cc:1467] Processing CreateTablet for tablet 1d8ba8971e82402c95e02ca10dee1fb6 (DEFAULT_TABLE table=test-table [id=63a27bbb37cb42e3b4c6bc7f5f86584d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:53:39.340802 15547 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1d8ba8971e82402c95e02ca10dee1fb6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:53:39.343087 15634 tablet_bootstrap.cc:492] T 1d8ba8971e82402c95e02ca10dee1fb6 P 67bb7fd16e8b4d60b9e82d20acffab4e: Bootstrap starting.
I20260806 22:53:39.343537 15635 tablet_bootstrap.cc:492] T 1d8ba8971e82402c95e02ca10dee1fb6 P b1ee65c704e642499e51462281b5d15c: Bootstrap starting.
I20260806 22:53:39.343792 15634 tablet_bootstrap.cc:654] T 1d8ba8971e82402c95e02ca10dee1fb6 P 67bb7fd16e8b4d60b9e82d20acffab4e: Neither blocks nor log segments found. Creating new log.
I20260806 22:53:39.343830 15633 tablet_bootstrap.cc:654] T 1d8ba8971e82402c95e02ca10dee1fb6 P 18509d307ad44c9ba4d309e2c15da3bb: Neither blocks nor log segments found. Creating new log.
I20260806 22:53:39.344300 15635 tablet_bootstrap.cc:654] T 1d8ba8971e82402c95e02ca10dee1fb6 P b1ee65c704e642499e51462281b5d15c: Neither blocks nor log segments found. Creating new log.
I20260806 22:53:39.344329 15634 log.cc:824] T 1d8ba8971e82402c95e02ca10dee1fb6 P 67bb7fd16e8b4d60b9e82d20acffab4e: Log is configured to *not* fsync() on all Append() calls
I20260806 22:53:39.344429 15633 log.cc:824] T 1d8ba8971e82402c95e02ca10dee1fb6 P 18509d307ad44c9ba4d309e2c15da3bb: Log is configured to *not* fsync() on all Append() calls
I20260806 22:53:39.344861 15635 log.cc:824] T 1d8ba8971e82402c95e02ca10dee1fb6 P b1ee65c704e642499e51462281b5d15c: Log is configured to *not* fsync() on all Append() calls
I20260806 22:53:39.345223 15633 tablet_bootstrap.cc:492] T 1d8ba8971e82402c95e02ca10dee1fb6 P 18509d307ad44c9ba4d309e2c15da3bb: No bootstrap required, opened a new log
I20260806 22:53:39.345225 15634 tablet_bootstrap.cc:492] T 1d8ba8971e82402c95e02ca10dee1fb6 P 67bb7fd16e8b4d60b9e82d20acffab4e: No bootstrap required, opened a new log
I20260806 22:53:39.345293 15634 ts_tablet_manager.cc:1397] T 1d8ba8971e82402c95e02ca10dee1fb6 P 67bb7fd16e8b4d60b9e82d20acffab4e: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260806 22:53:39.345310 15633 ts_tablet_manager.cc:1397] T 1d8ba8971e82402c95e02ca10dee1fb6 P 18509d307ad44c9ba4d309e2c15da3bb: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260806 22:53:39.345629 15635 tablet_bootstrap.cc:492] T 1d8ba8971e82402c95e02ca10dee1fb6 P b1ee65c704e642499e51462281b5d15c: No bootstrap required, opened a new log
I20260806 22:53:39.345714 15635 ts_tablet_manager.cc:1397] T 1d8ba8971e82402c95e02ca10dee1fb6 P b1ee65c704e642499e51462281b5d15c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:53:39.346812 15633 raft_consensus.cc:348] T 1d8ba8971e82402c95e02ca10dee1fb6 P 18509d307ad44c9ba4d309e2c15da3bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67bb7fd16e8b4d60b9e82d20acffab4e" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 40633 } } peers { permanent_uuid: "18509d307ad44c9ba4d309e2c15da3bb" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 42119 } } peers { permanent_uuid: "b1ee65c704e642499e51462281b5d15c" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 46717 } }
I20260806 22:53:39.346935 15633 raft_consensus.cc:374] T 1d8ba8971e82402c95e02ca10dee1fb6 P 18509d307ad44c9ba4d309e2c15da3bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:53:39.346956 15633 raft_consensus.cc:729] T 1d8ba8971e82402c95e02ca10dee1fb6 P 18509d307ad44c9ba4d309e2c15da3bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18509d307ad44c9ba4d309e2c15da3bb, State: Initialized, Role: FOLLOWER
I20260806 22:53:39.347077 15633 consensus_queue.cc:260] T 1d8ba8971e82402c95e02ca10dee1fb6 P 18509d307ad44c9ba4d309e2c15da3bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67bb7fd16e8b4d60b9e82d20acffab4e" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 40633 } } peers { permanent_uuid: "18509d307ad44c9ba4d309e2c15da3bb" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 42119 } } peers { permanent_uuid: "b1ee65c704e642499e51462281b5d15c" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 46717 } }
I20260806 22:53:39.347357 15633 ts_tablet_manager.cc:1428] T 1d8ba8971e82402c95e02ca10dee1fb6 P 18509d307ad44c9ba4d309e2c15da3bb: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:53:39.347309 15634 raft_consensus.cc:348] T 1d8ba8971e82402c95e02ca10dee1fb6 P 67bb7fd16e8b4d60b9e82d20acffab4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67bb7fd16e8b4d60b9e82d20acffab4e" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 40633 } } peers { permanent_uuid: "18509d307ad44c9ba4d309e2c15da3bb" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 42119 } } peers { permanent_uuid: "b1ee65c704e642499e51462281b5d15c" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 46717 } }
I20260806 22:53:39.347487 15472 heartbeater.cc:503] Master 127.28.218.254:37943 was elected leader, sending a full tablet report...
I20260806 22:53:39.347498 15634 raft_consensus.cc:374] T 1d8ba8971e82402c95e02ca10dee1fb6 P 67bb7fd16e8b4d60b9e82d20acffab4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:53:39.347539 15634 raft_consensus.cc:729] T 1d8ba8971e82402c95e02ca10dee1fb6 P 67bb7fd16e8b4d60b9e82d20acffab4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67bb7fd16e8b4d60b9e82d20acffab4e, State: Initialized, Role: FOLLOWER
I20260806 22:53:39.347643 15634 consensus_queue.cc:260] T 1d8ba8971e82402c95e02ca10dee1fb6 P 67bb7fd16e8b4d60b9e82d20acffab4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67bb7fd16e8b4d60b9e82d20acffab4e" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 40633 } } peers { permanent_uuid: "18509d307ad44c9ba4d309e2c15da3bb" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 42119 } } peers { permanent_uuid: "b1ee65c704e642499e51462281b5d15c" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 46717 } }
I20260806 22:53:39.347612 15635 raft_consensus.cc:348] T 1d8ba8971e82402c95e02ca10dee1fb6 P b1ee65c704e642499e51462281b5d15c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67bb7fd16e8b4d60b9e82d20acffab4e" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 40633 } } peers { permanent_uuid: "18509d307ad44c9ba4d309e2c15da3bb" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 42119 } } peers { permanent_uuid: "b1ee65c704e642499e51462281b5d15c" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 46717 } }
I20260806 22:53:39.347769 15635 raft_consensus.cc:374] T 1d8ba8971e82402c95e02ca10dee1fb6 P b1ee65c704e642499e51462281b5d15c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:53:39.347810 15635 raft_consensus.cc:729] T 1d8ba8971e82402c95e02ca10dee1fb6 P b1ee65c704e642499e51462281b5d15c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1ee65c704e642499e51462281b5d15c, State: Initialized, Role: FOLLOWER
I20260806 22:53:39.347898 15634 ts_tablet_manager.cc:1428] T 1d8ba8971e82402c95e02ca10dee1fb6 P 67bb7fd16e8b4d60b9e82d20acffab4e: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260806 22:53:39.347920 15635 consensus_queue.cc:260] T 1d8ba8971e82402c95e02ca10dee1fb6 P b1ee65c704e642499e51462281b5d15c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67bb7fd16e8b4d60b9e82d20acffab4e" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 40633 } } peers { permanent_uuid: "18509d307ad44c9ba4d309e2c15da3bb" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 42119 } } peers { permanent_uuid: "b1ee65c704e642499e51462281b5d15c" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 46717 } }
I20260806 22:53:39.348028 15327 heartbeater.cc:503] Master 127.28.218.254:37943 was elected leader, sending a full tablet report...
I20260806 22:53:39.348203 15635 ts_tablet_manager.cc:1428] T 1d8ba8971e82402c95e02ca10dee1fb6 P b1ee65c704e642499e51462281b5d15c: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 22:53:39.348356 15615 heartbeater.cc:503] Master 127.28.218.254:37943 was elected leader, sending a full tablet report...
W20260806 22:53:39.401257 15473 tablet.cc:2369] T 1d8ba8971e82402c95e02ca10dee1fb6 P 18509d307ad44c9ba4d309e2c15da3bb: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 22:53:39.488981 15328 tablet.cc:2369] T 1d8ba8971e82402c95e02ca10dee1fb6 P 67bb7fd16e8b4d60b9e82d20acffab4e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 22:53:39.549576 15616 tablet.cc:2369] T 1d8ba8971e82402c95e02ca10dee1fb6 P b1ee65c704e642499e51462281b5d15c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:53:39.607234 15640 raft_consensus.cc:482] T 1d8ba8971e82402c95e02ca10dee1fb6 P 67bb7fd16e8b4d60b9e82d20acffab4e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:53:39.607378 15640 raft_consensus.cc:504] T 1d8ba8971e82402c95e02ca10dee1fb6 P 67bb7fd16e8b4d60b9e82d20acffab4e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67bb7fd16e8b4d60b9e82d20acffab4e" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 40633 } } peers { permanent_uuid: "18509d307ad44c9ba4d309e2c15da3bb" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 42119 } } peers { permanent_uuid: "b1ee65c704e642499e51462281b5d15c" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 46717 } }
I20260806 22:53:39.607805 15640 leader_election.cc:288] T 1d8ba8971e82402c95e02ca10dee1fb6 P 67bb7fd16e8b4d60b9e82d20acffab4e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 18509d307ad44c9ba4d309e2c15da3bb (127.28.218.194:42119), b1ee65c704e642499e51462281b5d15c (127.28.218.195:46717)
I20260806 22:53:39.612020 15632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:39.608337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.193:59493 (local address 127.28.218.195:46717)
0806 22:53:39.608388 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0806 22:53:39.608392 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:53:39.608409 (+ 17us) server_negotiation.cc:407] Connection header received
0806 22:53:39.608476 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:53:39.608477 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:53:39.608485 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:53:39.608544 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:53:39.608758 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:39.609506 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:39.611022 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:39.611624 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:39.611704 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:39.611713 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 22:53:39.611907 (+ 194us) server_negotiation.cc:299] Negotiation successful
0806 22:53:39.611945 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 22:53:39.612017 15630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:39.608109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.193:39311 (local address 127.28.218.194:42119)
0806 22:53:39.608189 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0806 22:53:39.608191 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:53:39.608292 (+ 101us) server_negotiation.cc:407] Connection header received
0806 22:53:39.608452 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:53:39.608453 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:53:39.608466 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:53:39.608544 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:53:39.608754 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:39.609512 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:39.611016 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:39.611624 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:39.611704 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:39.611713 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 22:53:39.611907 (+ 194us) server_negotiation.cc:299] Negotiation successful
0806 22:53:39.611945 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 22:53:39.612049 15642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:39.607994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.194:42119 (local address 127.28.218.193:39311)
0806 22:53:39.608192 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0806 22:53:39.608205 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:53:39.608306 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:53:39.608559 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:53:39.608560 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:53:39.608562 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:53:39.608662 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:53:39.608665 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:53:39.609604 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:53:39.609605 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:53:39.610916 (+ 1311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:53:39.610919 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:53:39.611718 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:53:39.611720 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:53:39.611807 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:39.611813 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 22:53:39.611876 (+ 63us) client_negotiation.cc:240] Negotiation successful
0806 22:53:39.611936 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"mutex_wait_us":117,"thread_start_us":119,"threads_started":1}
I20260806 22:53:39.612049 15643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:39.608256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.195:46717 (local address 127.28.218.193:59493)
0806 22:53:39.608360 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0806 22:53:39.608370 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:53:39.608415 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:53:39.608559 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:53:39.608559 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:53:39.608566 (+ 7us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:53:39.608673 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:53:39.608675 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:53:39.609605 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:53:39.609606 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:53:39.610916 (+ 1310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:53:39.610919 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:53:39.611720 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:53:39.611721 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:53:39.611807 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:39.611815 (+ 8us) client_negotiation.cc:769] Sending connection context
0806 22:53:39.611876 (+ 61us) client_negotiation.cc:240] Negotiation successful
0806 22:53:39.611936 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":91,"mutex_wait_us":47,"thread_start_us":49,"threads_started":1}
I20260806 22:53:39.612710 15567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d8ba8971e82402c95e02ca10dee1fb6" candidate_uuid: "67bb7fd16e8b4d60b9e82d20acffab4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1ee65c704e642499e51462281b5d15c" is_pre_election: true
I20260806 22:53:39.612710 15422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d8ba8971e82402c95e02ca10dee1fb6" candidate_uuid: "67bb7fd16e8b4d60b9e82d20acffab4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18509d307ad44c9ba4d309e2c15da3bb" is_pre_election: true
I20260806 22:53:39.612883 15567 raft_consensus.cc:2454] T 1d8ba8971e82402c95e02ca10dee1fb6 P b1ee65c704e642499e51462281b5d15c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 67bb7fd16e8b4d60b9e82d20acffab4e in term 0.
I20260806 22:53:39.612883 15422 raft_consensus.cc:2454] T 1d8ba8971e82402c95e02ca10dee1fb6 P 18509d307ad44c9ba4d309e2c15da3bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 67bb7fd16e8b4d60b9e82d20acffab4e in term 0.
I20260806 22:53:39.613116 15208 leader_election.cc:302] T 1d8ba8971e82402c95e02ca10dee1fb6 P 67bb7fd16e8b4d60b9e82d20acffab4e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 67bb7fd16e8b4d60b9e82d20acffab4e, b1ee65c704e642499e51462281b5d15c; no voters:
I20260806 22:53:39.613261 15640 raft_consensus.cc:2781] T 1d8ba8971e82402c95e02ca10dee1fb6 P 67bb7fd16e8b4d60b9e82d20acffab4e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:53:39.613344 15640 raft_consensus.cc:482] T 1d8ba8971e82402c95e02ca10dee1fb6 P 67bb7fd16e8b4d60b9e82d20acffab4e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:53:39.613380 15640 raft_consensus.cc:3037] T 1d8ba8971e82402c95e02ca10dee1fb6 P 67bb7fd16e8b4d60b9e82d20acffab4e [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:53:39.614111 15640 raft_consensus.cc:504] T 1d8ba8971e82402c95e02ca10dee1fb6 P 67bb7fd16e8b4d60b9e82d20acffab4e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67bb7fd16e8b4d60b9e82d20acffab4e" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 40633 } } peers { permanent_uuid: "18509d307ad44c9ba4d309e2c15da3bb" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 42119 } } peers { permanent_uuid: "b1ee65c704e642499e51462281b5d15c" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 46717 } }
I20260806 22:53:39.614252 15640 leader_election.cc:288] T 1d8ba8971e82402c95e02ca10dee1fb6 P 67bb7fd16e8b4d60b9e82d20acffab4e [CANDIDATE]: Term 1 election: Requested vote from peers 18509d307ad44c9ba4d309e2c15da3bb (127.28.218.194:42119), b1ee65c704e642499e51462281b5d15c (127.28.218.195:46717)
I20260806 22:53:39.614437 15422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d8ba8971e82402c95e02ca10dee1fb6" candidate_uuid: "67bb7fd16e8b4d60b9e82d20acffab4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18509d307ad44c9ba4d309e2c15da3bb"
I20260806 22:53:39.614466 15567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d8ba8971e82402c95e02ca10dee1fb6" candidate_uuid: "67bb7fd16e8b4d60b9e82d20acffab4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1ee65c704e642499e51462281b5d15c"
I20260806 22:53:39.614524 15422 raft_consensus.cc:3037] T 1d8ba8971e82402c95e02ca10dee1fb6 P 18509d307ad44c9ba4d309e2c15da3bb [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:53:39.614570 15567 raft_consensus.cc:3037] T 1d8ba8971e82402c95e02ca10dee1fb6 P b1ee65c704e642499e51462281b5d15c [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:53:39.615378 15567 raft_consensus.cc:2454] T 1d8ba8971e82402c95e02ca10dee1fb6 P b1ee65c704e642499e51462281b5d15c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 67bb7fd16e8b4d60b9e82d20acffab4e in term 1.
I20260806 22:53:39.615432 15422 raft_consensus.cc:2454] T 1d8ba8971e82402c95e02ca10dee1fb6 P 18509d307ad44c9ba4d309e2c15da3bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 67bb7fd16e8b4d60b9e82d20acffab4e in term 1.
I20260806 22:53:39.615633 15208 leader_election.cc:302] T 1d8ba8971e82402c95e02ca10dee1fb6 P 67bb7fd16e8b4d60b9e82d20acffab4e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18509d307ad44c9ba4d309e2c15da3bb, 67bb7fd16e8b4d60b9e82d20acffab4e; no voters:
I20260806 22:53:39.615759 15640 raft_consensus.cc:2781] T 1d8ba8971e82402c95e02ca10dee1fb6 P 67bb7fd16e8b4d60b9e82d20acffab4e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:53:39.615900 15640 raft_consensus.cc:686] T 1d8ba8971e82402c95e02ca10dee1fb6 P 67bb7fd16e8b4d60b9e82d20acffab4e [term 1 LEADER]: Becoming Leader. State: Replica: 67bb7fd16e8b4d60b9e82d20acffab4e, State: Running, Role: LEADER
I20260806 22:53:39.615983 15640 consensus_queue.cc:237] T 1d8ba8971e82402c95e02ca10dee1fb6 P 67bb7fd16e8b4d60b9e82d20acffab4e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67bb7fd16e8b4d60b9e82d20acffab4e" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 40633 } } peers { permanent_uuid: "18509d307ad44c9ba4d309e2c15da3bb" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 42119 } } peers { permanent_uuid: "b1ee65c704e642499e51462281b5d15c" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 46717 } }
I20260806 22:53:39.616809 15126 catalog_manager.cc:5495] T 1d8ba8971e82402c95e02ca10dee1fb6 P 67bb7fd16e8b4d60b9e82d20acffab4e reported cstate change: term changed from 0 to 1, leader changed from <none> to 67bb7fd16e8b4d60b9e82d20acffab4e (127.28.218.193). New cstate: current_term: 1 leader_uuid: "67bb7fd16e8b4d60b9e82d20acffab4e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67bb7fd16e8b4d60b9e82d20acffab4e" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 40633 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "18509d307ad44c9ba4d309e2c15da3bb" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 42119 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1ee65c704e642499e51462281b5d15c" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 46717 } health_report { overall_health: UNKNOWN } } }
I20260806 22:53:39.690362 15628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:39.688186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52312 (local address 127.28.218.193:40633)
0806 22:53:39.688228 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 22:53:39.688230 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:53:39.688266 (+ 36us) server_negotiation.cc:407] Connection header received
0806 22:53:39.688302 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:53:39.688303 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:53:39.688323 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:53:39.688376 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:53:39.688507 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:39.688968 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:39.689721 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:39.689897 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:39.689948 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:39.690092 (+ 144us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:53:39.690191 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:53:39.690251 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 22:53:39.690284 (+ 33us) server_negotiation.cc:299] Negotiation successful
0806 22:53:39.690303 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 22:53:39.692315 15567 raft_consensus.cc:1261] T 1d8ba8971e82402c95e02ca10dee1fb6 P b1ee65c704e642499e51462281b5d15c [term 1 FOLLOWER]: Refusing update from remote peer 67bb7fd16e8b4d60b9e82d20acffab4e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:53:39.692341 15422 raft_consensus.cc:1261] T 1d8ba8971e82402c95e02ca10dee1fb6 P 18509d307ad44c9ba4d309e2c15da3bb [term 1 FOLLOWER]: Refusing update from remote peer 67bb7fd16e8b4d60b9e82d20acffab4e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:53:39.692683 15640 consensus_queue.cc:1035] T 1d8ba8971e82402c95e02ca10dee1fb6 P 67bb7fd16e8b4d60b9e82d20acffab4e [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1ee65c704e642499e51462281b5d15c" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 46717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:53:39.692842 15644 consensus_queue.cc:1035] T 1d8ba8971e82402c95e02ca10dee1fb6 P 67bb7fd16e8b4d60b9e82d20acffab4e [LEADER]: Connected to new peer: Peer: permanent_uuid: "18509d307ad44c9ba4d309e2c15da3bb" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 42119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:53:39.695526 15649 mvcc.cc:205] Tried to move back new op lower bound from 7315688733456838656 to 7315688733147648000. Current Snapshot: MvccSnapshot[applied={T|T < 7315688733456838656}]
I20260806 22:53:39.695786 15647 mvcc.cc:205] Tried to move back new op lower bound from 7315688733456838656 to 7315688733147648000. Current Snapshot: MvccSnapshot[applied={T|T < 7315688733456838656}]
I20260806 22:53:39.698225 15126 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55974:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:53:39.698318 15126 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55974:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:53:39.699102 15126 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3ad431b9c0cf44c383918410d4f3e7c3: Sending DeleteTablet for 3 replicas of tablet 1d8ba8971e82402c95e02ca10dee1fb6
I20260806 22:53:39.699488 15257 tablet_service.cc:1514] Processing DeleteTablet for tablet 1d8ba8971e82402c95e02ca10dee1fb6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:53:39 UTC) from {username='kudu', principal='kudu/127.28.218.254@KRBTEST.COM'} at 127.0.0.1:60122
I20260806 22:53:39.699574 15547 tablet_service.cc:1514] Processing DeleteTablet for tablet 1d8ba8971e82402c95e02ca10dee1fb6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:53:39 UTC) from {username='kudu', principal='kudu/127.28.218.254@KRBTEST.COM'} at 127.0.0.1:33602
I20260806 22:53:39.699741 15657 tablet_replica.cc:330] T 1d8ba8971e82402c95e02ca10dee1fb6 P 67bb7fd16e8b4d60b9e82d20acffab4e: stopping tablet replica
I20260806 22:53:39.699756 15400 tablet_service.cc:1514] Processing DeleteTablet for tablet 1d8ba8971e82402c95e02ca10dee1fb6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:53:39 UTC) from {username='kudu', principal='kudu/127.28.218.254@KRBTEST.COM'} at 127.0.0.1:43862
I20260806 22:53:39.699847 15657 raft_consensus.cc:2229] T 1d8ba8971e82402c95e02ca10dee1fb6 P 67bb7fd16e8b4d60b9e82d20acffab4e [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:53:39.699965 15657 raft_consensus.cc:2258] T 1d8ba8971e82402c95e02ca10dee1fb6 P 67bb7fd16e8b4d60b9e82d20acffab4e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:53:39.700132 15659 tablet_replica.cc:330] T 1d8ba8971e82402c95e02ca10dee1fb6 P 18509d307ad44c9ba4d309e2c15da3bb: stopping tablet replica
I20260806 22:53:39.700132 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 15190
I20260806 22:53:39.701000 15659 raft_consensus.cc:2229] T 1d8ba8971e82402c95e02ca10dee1fb6 P 18509d307ad44c9ba4d309e2c15da3bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:53:39.701166 15659 raft_consensus.cc:2258] T 1d8ba8971e82402c95e02ca10dee1fb6 P 18509d307ad44c9ba4d309e2c15da3bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:53:39.701436 15658 tablet_replica.cc:330] T 1d8ba8971e82402c95e02ca10dee1fb6 P b1ee65c704e642499e51462281b5d15c: stopping tablet replica
I20260806 22:53:39.701736 15659 ts_tablet_manager.cc:1905] T 1d8ba8971e82402c95e02ca10dee1fb6 P 18509d307ad44c9ba4d309e2c15da3bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:53:39.703435 15658 raft_consensus.cc:2229] T 1d8ba8971e82402c95e02ca10dee1fb6 P b1ee65c704e642499e51462281b5d15c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:53:39.703588 15658 raft_consensus.cc:2258] T 1d8ba8971e82402c95e02ca10dee1fb6 P b1ee65c704e642499e51462281b5d15c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:53:39.703924 15659 ts_tablet_manager.cc:1918] T 1d8ba8971e82402c95e02ca10dee1fb6 P 18509d307ad44c9ba4d309e2c15da3bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:53:39.704011 15659 log.cc:1196] T 1d8ba8971e82402c95e02ca10dee1fb6 P 18509d307ad44c9ba4d309e2c15da3bb: Deleting WAL directory at /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056586788339-29547-0/minicluster-data/ts-1/wal/wals/1d8ba8971e82402c95e02ca10dee1fb6
I20260806 22:53:39.704057 15658 ts_tablet_manager.cc:1905] T 1d8ba8971e82402c95e02ca10dee1fb6 P b1ee65c704e642499e51462281b5d15c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:53:39.704335 15659 ts_tablet_manager.cc:1939] T 1d8ba8971e82402c95e02ca10dee1fb6 P 18509d307ad44c9ba4d309e2c15da3bb: Deleting consensus metadata
I20260806 22:53:39.704756 15109 catalog_manager.cc:4841] TS 18509d307ad44c9ba4d309e2c15da3bb (127.28.218.194:42119): tablet 1d8ba8971e82402c95e02ca10dee1fb6 (table test-table [id=63a27bbb37cb42e3b4c6bc7f5f86584d]) successfully deleted
I20260806 22:53:39.705696 15658 ts_tablet_manager.cc:1918] T 1d8ba8971e82402c95e02ca10dee1fb6 P b1ee65c704e642499e51462281b5d15c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:53:39.705755 15658 log.cc:1196] T 1d8ba8971e82402c95e02ca10dee1fb6 P b1ee65c704e642499e51462281b5d15c: Deleting WAL directory at /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056586788339-29547-0/minicluster-data/ts-2/wal/wals/1d8ba8971e82402c95e02ca10dee1fb6
I20260806 22:53:39.706034 15658 ts_tablet_manager.cc:1939] T 1d8ba8971e82402c95e02ca10dee1fb6 P b1ee65c704e642499e51462281b5d15c: Deleting consensus metadata
W20260806 22:53:39.706590 15109 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.28.218.193:40633 (error 108)
I20260806 22:53:39.706807 15109 catalog_manager.cc:4841] TS b1ee65c704e642499e51462281b5d15c (127.28.218.195:46717): tablet 1d8ba8971e82402c95e02ca10dee1fb6 (table test-table [id=63a27bbb37cb42e3b4c6bc7f5f86584d]) successfully deleted
I20260806 22:53:39.706847 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 15332
I20260806 22:53:39.707078 15629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:39.706952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.193:40633 (local address 127.0.0.1:52316)
0806 22:53:39.707001 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0806 22:53:39.707043 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.193:40633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260806 22:53:39.707841 15109 catalog_manager.cc:4584] TS 67bb7fd16e8b4d60b9e82d20acffab4e (127.28.218.193:40633): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1d8ba8971e82402c95e02ca10dee1fb6: Network error: Client connection negotiation failed: client connection to 127.28.218.193:40633: connect: Connection refused (error 111)
I20260806 22:53:39.717839 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 15477
I20260806 22:53:39.722970 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 15093
2026-08-06T22:53:39Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2248 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-06T22:53:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:53:39Z Disabled control of system clock
I20260806 22:53:39.752266 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056586788339-29547-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056586788339-29547-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056586788339-29547-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.218.254:42877
--webserver_interface=127.28.218.254
--webserver_port=0
--builtin_ntp_servers=127.28.218.212:45793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.218.254:42877
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 22:53:39.832264 15664 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:53:39.832482 15664 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:53:39.832504 15664 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:53:39.833564 15664 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:53:39.833596 15664 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:53:39.833608 15664 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:53:39.833621 15664 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:53:39.833633 15664 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:53:39.834872 15664 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:45793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056586788339-29547-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056586788339-29547-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.218.254:42877
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.218.254:42877
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.218.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15664
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056586788339-29547-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:53:39.835049 15664 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:53:39.835372 15664 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:53:39.840343 15669 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:53:39.840325 15670 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:53:39.840317 15672 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:53:39.840446 15664 server_base.cc:1034] running on GCE node
I20260806 22:53:39.840934 15664 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:53:39.841239 15664 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:53:39.842406 15664 hybrid_clock.cc:648] HybridClock initialized: now 1786056819842394 us; error 47 us; skew 500 ppm
I20260806 22:53:39.843832 15664 webserver.cc:460] Webserver started at http://127.28.218.254:36103/ using document root <none> and password file <none>
I20260806 22:53:39.844098 15664 fs_manager.cc:359] Metadata directory not provided
I20260806 22:53:39.844177 15664 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:53:39.844333 15664 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:53:39.845407 15664 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056586788339-29547-0/minicluster-data/master-0/data/instance:
uuid: "bd1d36cf82604aabb3df0becaaab2b3d"
format_stamp: "Formatted at 2026-08-06 22:53:39 on dist-test-slave-1xrh"
server_key: "0f32f514a127840350175ec2931bc564"
server_key_iv: "f75867bccf83afd4e508b0e755537453"
server_key_version: "encryptionkey@0"
I20260806 22:53:39.845777 15664 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056586788339-29547-0/minicluster-data/master-0/wal/instance:
uuid: "bd1d36cf82604aabb3df0becaaab2b3d"
format_stamp: "Formatted at 2026-08-06 22:53:39 on dist-test-slave-1xrh"
server_key: "0f32f514a127840350175ec2931bc564"
server_key_iv: "f75867bccf83afd4e508b0e755537453"
server_key_version: "encryptionkey@0"
I20260806 22:53:39.847110 15664 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 22:53:39.847858 15678 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:53:39.848026 15664 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:53:39.848129 15664 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056586788339-29547-0/minicluster-data/master-0/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056586788339-29547-0/minicluster-data/master-0/wal
uuid: "bd1d36cf82604aabb3df0becaaab2b3d"
format_stamp: "Formatted at 2026-08-06 22:53:39 on dist-test-slave-1xrh"
server_key: "0f32f514a127840350175ec2931bc564"
server_key_iv: "f75867bccf83afd4e508b0e755537453"
server_key_version: "encryptionkey@0"
I20260806 22:53:39.848215 15664 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056586788339-29547-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056586788339-29547-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056586788339-29547-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:53:39.882376 15664 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:53:39.882787 15664 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:53:39.882961 15664 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:53:39.887032 15664 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.254:42877
I20260806 22:53:39.887065 15732 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.254:42877 every 8 connection(s)
I20260806 22:53:39.887450 15664 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
I20260806 22:53:39.888411 15733 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:53:39.890707 15733 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd1d36cf82604aabb3df0becaaab2b3d: Bootstrap starting.
I20260806 22:53:39.891371 15733 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bd1d36cf82604aabb3df0becaaab2b3d: Neither blocks nor log segments found. Creating new log.
I20260806 22:53:39.891674 15733 log.cc:824] T 00000000000000000000000000000000 P bd1d36cf82604aabb3df0becaaab2b3d: Log is configured to *not* fsync() on all Append() calls
I20260806 22:53:39.892354 15733 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd1d36cf82604aabb3df0becaaab2b3d: No bootstrap required, opened a new log
I20260806 22:53:39.893988 15733 raft_consensus.cc:348] T 00000000000000000000000000000000 P bd1d36cf82604aabb3df0becaaab2b3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd1d36cf82604aabb3df0becaaab2b3d" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 42877 } }
I20260806 22:53:39.894130 15733 raft_consensus.cc:374] T 00000000000000000000000000000000 P bd1d36cf82604aabb3df0becaaab2b3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:53:39.894150 15733 raft_consensus.cc:729] T 00000000000000000000000000000000 P bd1d36cf82604aabb3df0becaaab2b3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd1d36cf82604aabb3df0becaaab2b3d, State: Initialized, Role: FOLLOWER
I20260806 22:53:39.894336 15733 consensus_queue.cc:260] T 00000000000000000000000000000000 P bd1d36cf82604aabb3df0becaaab2b3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd1d36cf82604aabb3df0becaaab2b3d" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 42877 } }
I20260806 22:53:39.894410 15733 raft_consensus.cc:388] T 00000000000000000000000000000000 P bd1d36cf82604aabb3df0becaaab2b3d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:53:39.894452 15733 raft_consensus.cc:482] T 00000000000000000000000000000000 P bd1d36cf82604aabb3df0becaaab2b3d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:53:39.894516 15733 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bd1d36cf82604aabb3df0becaaab2b3d [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:53:39.895092 15733 raft_consensus.cc:504] T 00000000000000000000000000000000 P bd1d36cf82604aabb3df0becaaab2b3d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd1d36cf82604aabb3df0becaaab2b3d" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 42877 } }
I20260806 22:53:39.895265 15733 leader_election.cc:302] T 00000000000000000000000000000000 P bd1d36cf82604aabb3df0becaaab2b3d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bd1d36cf82604aabb3df0becaaab2b3d; no voters:
I20260806 22:53:39.895430 15733 leader_election.cc:288] T 00000000000000000000000000000000 P bd1d36cf82604aabb3df0becaaab2b3d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:53:39.895498 15736 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bd1d36cf82604aabb3df0becaaab2b3d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:53:39.895659 15733 sys_catalog.cc:564] T 00000000000000000000000000000000 P bd1d36cf82604aabb3df0becaaab2b3d [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:53:39.895692 15736 raft_consensus.cc:686] T 00000000000000000000000000000000 P bd1d36cf82604aabb3df0becaaab2b3d [term 1 LEADER]: Becoming Leader. State: Replica: bd1d36cf82604aabb3df0becaaab2b3d, State: Running, Role: LEADER
I20260806 22:53:39.895812 15736 consensus_queue.cc:237] T 00000000000000000000000000000000 P bd1d36cf82604aabb3df0becaaab2b3d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd1d36cf82604aabb3df0becaaab2b3d" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 42877 } }
I20260806 22:53:39.896304 15737 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd1d36cf82604aabb3df0becaaab2b3d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bd1d36cf82604aabb3df0becaaab2b3d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd1d36cf82604aabb3df0becaaab2b3d" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 42877 } } }
I20260806 22:53:39.896389 15737 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd1d36cf82604aabb3df0becaaab2b3d [sys.catalog]: This master's current role is: LEADER
I20260806 22:53:39.896481 15738 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd1d36cf82604aabb3df0becaaab2b3d [sys.catalog]: SysCatalogTable state changed. Reason: New leader bd1d36cf82604aabb3df0becaaab2b3d. Latest consensus state: current_term: 1 leader_uuid: "bd1d36cf82604aabb3df0becaaab2b3d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd1d36cf82604aabb3df0becaaab2b3d" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 42877 } } }
I20260806 22:53:39.896863 15738 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd1d36cf82604aabb3df0becaaab2b3d [sys.catalog]: This master's current role is: LEADER
I20260806 22:53:39.896968 15750 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:53:39.897183 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 15664
I20260806 22:53:39.897295 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056586788339-29547-0/minicluster-data/master-0/wal/instance
I20260806 22:53:39.897537 29547 external_mini_cluster.cc:1428] Setting key 2518df3e8b0dae297a3d74e8b931ef4e
I20260806 22:53:39.897689 15750 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:53:39.899334 15750 catalog_manager.cc:1347] Generated new cluster ID: 642611dce302424a882ec9cff6948b1d
I20260806 22:53:39.899389 15750 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:53:39.903033 15756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:39.898028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42702 (local address 127.28.218.254:42877)
0806 22:53:39.898254 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0806 22:53:39.898262 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:53:39.898301 (+ 39us) server_negotiation.cc:407] Connection header received
0806 22:53:39.898480 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:53:39.898487 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:53:39.898549 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:53:39.898683 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:53:39.899070 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:39.900114 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:39.900965 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:39.901386 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:39.901550 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:39.902334 (+ 784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:53:39.902338 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:53:39.902338 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:53:39.902524 (+ 186us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:53:39.902557 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:53:39.902568 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:53:39.902649 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 22:53:39.902724 (+ 75us) server_negotiation.cc:299] Negotiation successful
0806 22:53:39.902804 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":88,"threads_started":1}
I20260806 22:53:39.930200 15750 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:53:39.930925 15750 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:53:39.942975 15750 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bd1d36cf82604aabb3df0becaaab2b3d: Generated new TSK 0
I20260806 22:53:39.943186 15750 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:53:39.946203 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056586788339-29547-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056586788339-29547-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056586788339-29547-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056586788339-29547-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.218.193:0
--local_ip_for_outbound_sockets=127.28.218.193
--webserver_interface=127.28.218.193
--webserver_port=0
--tserver_master_addrs=127.28.218.254:42877
--builtin_ntp_servers=127.28.218.212:45793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 22:53:40.030386 15758 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:53:40.030635 15758 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:53:40.030675 15758 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:53:40.031980 15758 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:53:40.032063 15758 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:53:40.032106 15758 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.218.193
I20260806 22:53:40.033475 15758 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:45793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056586788339-29547-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056586788339-29547-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.218.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056586788339-29547-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.218.193
--webserver_port=0
--tserver_master_addrs=127.28.218.254:42877
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15758
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.218.193
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056586788339-29547-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:53:40.033768 15758 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:53:40.034094 15758 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:53:40.039278 15766 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:53:40.039261 15764 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:53:40.039322 15758 server_base.cc:1034] running on GCE node
W20260806 22:53:40.039249 15763 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:53:40.039695 15758 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:53:40.039911 15758 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:53:40.041051 15758 hybrid_clock.cc:648] HybridClock initialized: now 1786056820041045 us; error 32 us; skew 500 ppm
I20260806 22:53:40.042309 15758 webserver.cc:460] Webserver started at http://127.28.218.193:37023/ using document root <none> and password file <none>
I20260806 22:53:40.042637 15758 fs_manager.cc:359] Metadata directory not provided
I20260806 22:53:40.042729 15758 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:53:40.042891 15758 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:53:40.044054 15758 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056586788339-29547-0/minicluster-data/ts-0/data/instance:
uuid: "beb1dc1064cb40ac953e76c536fc9e8e"
format_stamp: "Formatted at 2026-08-06 22:53:40 on dist-test-slave-1xrh"
server_key: "e52634e65639790e5e12937a1a7a8313"
server_key_iv: "18f0390524fa5565658f92c95b2daeee"
server_key_version: "encryptionkey@0"
I20260806 22:53:40.044498 15758 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056586788339-29547-0/minicluster-data/ts-0/wal/instance:
uuid: "beb1dc1064cb40ac953e76c536fc9e8e"
format_stamp: "Formatted at 2026-08-06 22:53:40 on dist-test-slave-1xrh"
server_key: "e52634e65639790e5e12937a1a7a8313"
server_key_iv: "18f0390524fa5565658f92c95b2daeee"
server_key_version: "encryptionkey@0"
I20260806 22:53:40.045830 15758 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:53:40.046574 15773 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:53:40.046756 15758 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:53:40.046856 15758 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056586788339-29547-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056586788339-29547-0/minicluster-data/ts-0/wal
uuid: "beb1dc1064cb40ac953e76c536fc9e8e"
format_stamp: "Formatted at 2026-08-06 22:53:40 on dist-test-slave-1xrh"
server_key: "e52634e65639790e5e12937a1a7a8313"
server_key_iv: "18f0390524fa5565658f92c95b2daeee"
server_key_version: "encryptionkey@0"
I20260806 22:53:40.046947 15758 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056586788339-29547-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056586788339-29547-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056586788339-29547-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:53:40.065656 15758 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:53:40.065994 15758 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:53:40.066160 15758 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:53:40.066407 15758 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:53:40.066766 15758 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:53:40.066828 15758 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:53:40.066869 15758 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:53:40.066896 15758 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:53:40.072852 15758 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.193:34889
I20260806 22:53:40.072940 15888 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.193:34889 every 8 connection(s)
I20260806 22:53:40.073300 15758 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056586788339-29547-0/minicluster-data/ts-0/data/info.pb
I20260806 22:53:40.078315 15756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:40.074147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.193:39621 (local address 127.28.218.254:42877)
0806 22:53:40.074255 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0806 22:53:40.074257 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:53:40.074403 (+ 146us) server_negotiation.cc:407] Connection header received
0806 22:53:40.074859 (+ 456us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:53:40.074860 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:53:40.074872 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:53:40.074920 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:53:40.075566 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:40.076044 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:40.076971 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:40.077153 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:40.077204 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:40.078096 (+ 892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:53:40.078096 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:53:40.078097 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:53:40.078100 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:53:40.078111 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:53:40.078114 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:53:40.078151 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0806 22:53:40.078238 (+ 87us) server_negotiation.cc:299] Negotiation successful
0806 22:53:40.078257 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260806 22:53:40.078511 15891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:40.074072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:42877 (local address 127.28.218.193:39621)
0806 22:53:40.074271 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0806 22:53:40.074306 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 22:53:40.074778 (+ 472us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:53:40.074978 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:53:40.074981 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:53:40.074995 (+ 14us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:53:40.075499 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:53:40.075503 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:53:40.076131 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:53:40.076132 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:53:40.076891 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:53:40.076893 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:53:40.077235 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:53:40.077236 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:53:40.077341 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:40.077947 (+ 606us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:53:40.077948 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:53:40.077975 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:53:40.078007 (+ 32us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:53:40.078171 (+ 164us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:53:40.078175 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:53:40.078177 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 22:53:40.078227 (+ 50us) client_negotiation.cc:240] Negotiation successful
0806 22:53:40.078306 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":66,"threads_started":1}
I20260806 22:53:40.079105 15889 heartbeater.cc:347] Connected to a master server at 127.28.218.254:42877
I20260806 22:53:40.079243 15889 heartbeater.cc:464] Registering TS with master...
I20260806 22:53:40.079421 15889 heartbeater.cc:511] Master 127.28.218.254:42877 requested a full tablet report, sending...
I20260806 22:53:40.079970 15696 ts_manager.cc:194] Registered new tserver with Master: beb1dc1064cb40ac953e76c536fc9e8e (127.28.218.193:34889)
I20260806 22:53:40.080566 15696 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.28.218.193:39621
I20260806 22:53:40.082036 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 15758
I20260806 22:53:40.082162 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056586788339-29547-0/minicluster-data/ts-0/wal/instance
I20260806 22:53:40.082357 29547 external_mini_cluster.cc:1428] Setting key cf0c1ecc7c1353247438b9503050a939
I20260806 22:53:40.083340 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056586788339-29547-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056586788339-29547-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056586788339-29547-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056586788339-29547-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.218.194:0
--local_ip_for_outbound_sockets=127.28.218.194
--webserver_interface=127.28.218.194
--webserver_port=0
--tserver_master_addrs=127.28.218.254:42877
--builtin_ntp_servers=127.28.218.212:45793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 22:53:40.165151 15892 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:53:40.165354 15892 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:53:40.165375 15892 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:53:40.166539 15892 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:53:40.166581 15892 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:53:40.166606 15892 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.218.194
I20260806 22:53:40.168021 15892 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:45793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056586788339-29547-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056586788339-29547-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.218.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056586788339-29547-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.218.194
--webserver_port=0
--tserver_master_addrs=127.28.218.254:42877
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15892
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.218.194
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056586788339-29547-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:53:40.168274 15892 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:53:40.168591 15892 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 22:53:40.173764 15892 server_base.cc:1034] running on GCE node
W20260806 22:53:40.173724 15898 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:53:40.173736 15901 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:53:40.173744 15897 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:53:40.174139 15892 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:53:40.174389 15892 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:53:40.175554 15892 hybrid_clock.cc:648] HybridClock initialized: now 1786056820175545 us; error 57 us; skew 500 ppm
I20260806 22:53:40.176746 15892 webserver.cc:460] Webserver started at http://127.28.218.194:42855/ using document root <none> and password file <none>
I20260806 22:53:40.177007 15892 fs_manager.cc:359] Metadata directory not provided
I20260806 22:53:40.177084 15892 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:53:40.177241 15892 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:53:40.178308 15892 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056586788339-29547-0/minicluster-data/ts-1/data/instance:
uuid: "7e160ca4e8a848d09f702f6cb7659dc1"
format_stamp: "Formatted at 2026-08-06 22:53:40 on dist-test-slave-1xrh"
server_key: "62d0036b85e4c85ace290ddfad36bd0b"
server_key_iv: "9c2878233ce76289f314302e3692e503"
server_key_version: "encryptionkey@0"
I20260806 22:53:40.178669 15892 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056586788339-29547-0/minicluster-data/ts-1/wal/instance:
uuid: "7e160ca4e8a848d09f702f6cb7659dc1"
format_stamp: "Formatted at 2026-08-06 22:53:40 on dist-test-slave-1xrh"
server_key: "62d0036b85e4c85ace290ddfad36bd0b"
server_key_iv: "9c2878233ce76289f314302e3692e503"
server_key_version: "encryptionkey@0"
I20260806 22:53:40.180002 15892 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:53:40.180708 15907 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:53:40.180893 15892 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:53:40.181003 15892 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056586788339-29547-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056586788339-29547-0/minicluster-data/ts-1/wal
uuid: "7e160ca4e8a848d09f702f6cb7659dc1"
format_stamp: "Formatted at 2026-08-06 22:53:40 on dist-test-slave-1xrh"
server_key: "62d0036b85e4c85ace290ddfad36bd0b"
server_key_iv: "9c2878233ce76289f314302e3692e503"
server_key_version: "encryptionkey@0"
I20260806 22:53:40.181102 15892 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056586788339-29547-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056586788339-29547-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056586788339-29547-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:53:40.198009 15892 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:53:40.198285 15892 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:53:40.198436 15892 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:53:40.198683 15892 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:53:40.199012 15892 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:53:40.199083 15892 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:53:40.199115 15892 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:53:40.199203 15892 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:53:40.205087 15892 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.194:33989
I20260806 22:53:40.205135 16024 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.194:33989 every 8 connection(s)
I20260806 22:53:40.205457 15892 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056586788339-29547-0/minicluster-data/ts-1/data/info.pb
I20260806 22:53:40.208725 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 15892
I20260806 22:53:40.208801 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056586788339-29547-0/minicluster-data/ts-1/wal/instance
I20260806 22:53:40.209012 29547 external_mini_cluster.cc:1428] Setting key 48fa2941afcee270e40327f5871c9721
I20260806 22:53:40.210117 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056586788339-29547-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056586788339-29547-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056586788339-29547-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056586788339-29547-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.218.195:0
--local_ip_for_outbound_sockets=127.28.218.195
--webserver_interface=127.28.218.195
--webserver_port=0
--tserver_master_addrs=127.28.218.254:42877
--builtin_ntp_servers=127.28.218.212:45793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 22:53:40.210920 15756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:40.206206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.194:50535 (local address 127.28.218.254:42877)
0806 22:53:40.206301 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0806 22:53:40.206304 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:53:40.206469 (+ 165us) server_negotiation.cc:407] Connection header received
0806 22:53:40.206997 (+ 528us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:53:40.206998 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:53:40.207009 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:53:40.207063 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:53:40.207746 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:40.208236 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:40.209453 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:40.209700 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:40.209774 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:40.210655 (+ 881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:53:40.210656 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:53:40.210657 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:53:40.210661 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:53:40.210675 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:53:40.210679 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:53:40.210723 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0806 22:53:40.210841 (+ 118us) server_negotiation.cc:299] Negotiation successful
0806 22:53:40.210864 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 22:53:40.211227 16027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:40.206115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:42877 (local address 127.28.218.194:50535)
0806 22:53:40.206293 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0806 22:53:40.206361 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0806 22:53:40.206881 (+ 520us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:53:40.207162 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:53:40.207165 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:53:40.207173 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:53:40.207662 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:53:40.207665 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:53:40.208321 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:53:40.208321 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:53:40.209350 (+ 1029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:53:40.209353 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:53:40.209807 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:53:40.209808 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:53:40.209964 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:40.210507 (+ 543us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:53:40.210507 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:53:40.210544 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:53:40.210569 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:53:40.210758 (+ 189us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:53:40.210762 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:53:40.210765 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:53:40.210841 (+ 76us) client_negotiation.cc:240] Negotiation successful
0806 22:53:40.210948 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":68,"threads_started":1}
I20260806 22:53:40.211853 16025 heartbeater.cc:347] Connected to a master server at 127.28.218.254:42877
I20260806 22:53:40.211973 16025 heartbeater.cc:464] Registering TS with master...
I20260806 22:53:40.212160 16025 heartbeater.cc:511] Master 127.28.218.254:42877 requested a full tablet report, sending...
I20260806 22:53:40.212596 15696 ts_manager.cc:194] Registered new tserver with Master: 7e160ca4e8a848d09f702f6cb7659dc1 (127.28.218.194:33989)
I20260806 22:53:40.212976 15696 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.28.218.194:50535
W20260806 22:53:40.293315 16028 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:53:40.293502 16028 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:53:40.293524 16028 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:53:40.294540 16028 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:53:40.294571 16028 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:53:40.294595 16028 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.218.195
I20260806 22:53:40.296134 16028 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:45793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056586788339-29547-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056586788339-29547-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.218.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056586788339-29547-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.218.195
--webserver_port=0
--tserver_master_addrs=127.28.218.254:42877
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16028
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.218.195
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056586788339-29547-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:53:40.296473 16028 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:53:40.296801 16028 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:53:40.301705 16033 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:53:40.301733 16034 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:53:40.301702 16036 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:53:40.301990 16028 server_base.cc:1034] running on GCE node
I20260806 22:53:40.302248 16028 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:53:40.302490 16028 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:53:40.303668 16028 hybrid_clock.cc:648] HybridClock initialized: now 1786056820303655 us; error 51 us; skew 500 ppm
I20260806 22:53:40.305087 16028 webserver.cc:460] Webserver started at http://127.28.218.195:33469/ using document root <none> and password file <none>
I20260806 22:53:40.305344 16028 fs_manager.cc:359] Metadata directory not provided
I20260806 22:53:40.305421 16028 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:53:40.305577 16028 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:53:40.306690 16028 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056586788339-29547-0/minicluster-data/ts-2/data/instance:
uuid: "f07b7eb6df6e478b844fd642f3473b63"
format_stamp: "Formatted at 2026-08-06 22:53:40 on dist-test-slave-1xrh"
server_key: "bab8bc95ce4e89261346285703a23e70"
server_key_iv: "aa4e05a13dc6f501f38ac4a2ea927f9d"
server_key_version: "encryptionkey@0"
I20260806 22:53:40.307067 16028 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056586788339-29547-0/minicluster-data/ts-2/wal/instance:
uuid: "f07b7eb6df6e478b844fd642f3473b63"
format_stamp: "Formatted at 2026-08-06 22:53:40 on dist-test-slave-1xrh"
server_key: "bab8bc95ce4e89261346285703a23e70"
server_key_iv: "aa4e05a13dc6f501f38ac4a2ea927f9d"
server_key_version: "encryptionkey@0"
I20260806 22:53:40.308363 16028 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:53:40.309036 16044 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:53:40.309218 16028 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:53:40.309314 16028 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056586788339-29547-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056586788339-29547-0/minicluster-data/ts-2/wal
uuid: "f07b7eb6df6e478b844fd642f3473b63"
format_stamp: "Formatted at 2026-08-06 22:53:40 on dist-test-slave-1xrh"
server_key: "bab8bc95ce4e89261346285703a23e70"
server_key_iv: "aa4e05a13dc6f501f38ac4a2ea927f9d"
server_key_version: "encryptionkey@0"
I20260806 22:53:40.309417 16028 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056586788339-29547-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056586788339-29547-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056586788339-29547-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:53:40.336963 16028 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:53:40.337330 16028 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:53:40.337503 16028 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:53:40.337761 16028 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:53:40.338150 16028 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:53:40.338213 16028 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:53:40.338249 16028 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:53:40.338276 16028 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:53:40.344285 16028 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.195:33461
I20260806 22:53:40.344353 16164 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.195:33461 every 8 connection(s)
I20260806 22:53:40.344727 16028 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056586788339-29547-0/minicluster-data/ts-2/data/info.pb
I20260806 22:53:40.346626 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 16028
I20260806 22:53:40.346740 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056586788339-29547-0/minicluster-data/ts-2/wal/instance
I20260806 22:53:40.346946 29547 external_mini_cluster.cc:1428] Setting key 909296bfe464a30c396c027d2988145a
I20260806 22:53:40.350536 15756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:40.345523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.195:56691 (local address 127.28.218.254:42877)
0806 22:53:40.345597 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0806 22:53:40.345599 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:53:40.345815 (+ 216us) server_negotiation.cc:407] Connection header received
0806 22:53:40.346309 (+ 494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:53:40.346310 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:53:40.346320 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:53:40.346382 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:53:40.347285 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:40.348093 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:40.349045 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:40.349230 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:40.349280 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:40.350259 (+ 979us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:53:40.350260 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:53:40.350261 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:53:40.350265 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:53:40.350277 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:53:40.350281 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:53:40.350351 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 22:53:40.350434 (+ 83us) server_negotiation.cc:299] Negotiation successful
0806 22:53:40.350456 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 22:53:40.350716 16167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:40.345491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:42877 (local address 127.28.218.195:56691)
0806 22:53:40.345679 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0806 22:53:40.345723 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 22:53:40.346222 (+ 499us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:53:40.346451 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:53:40.346454 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:53:40.346462 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:53:40.347182 (+ 720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:53:40.347189 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:53:40.348180 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:53:40.348181 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:53:40.348963 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:53:40.348965 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:53:40.349314 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:53:40.349315 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:53:40.349424 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:40.350125 (+ 701us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:53:40.350126 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:53:40.350155 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:53:40.350177 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:53:40.350350 (+ 173us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:53:40.350355 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:53:40.350357 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 22:53:40.350410 (+ 53us) client_negotiation.cc:240] Negotiation successful
0806 22:53:40.350490 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":90,"threads_started":1}
I20260806 22:53:40.351320 16165 heartbeater.cc:347] Connected to a master server at 127.28.218.254:42877
I20260806 22:53:40.351447 16165 heartbeater.cc:464] Registering TS with master...
I20260806 22:53:40.351663 16165 heartbeater.cc:511] Master 127.28.218.254:42877 requested a full tablet report, sending...
I20260806 22:53:40.352061 15696 ts_manager.cc:194] Registered new tserver with Master: f07b7eb6df6e478b844fd642f3473b63 (127.28.218.195:33461)
I20260806 22:53:40.352396 15696 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.28.218.195:56691
I20260806 22:53:40.358911 29547 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:53:40.363734 15756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:40.360403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42704 (local address 127.28.218.254:42877)
0806 22:53:40.360473 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0806 22:53:40.360475 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:53:40.360638 (+ 163us) server_negotiation.cc:407] Connection header received
0806 22:53:40.360714 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:53:40.360715 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:53:40.360726 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:53:40.360772 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:53:40.360983 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:40.361458 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:40.362216 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:40.362413 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:40.362496 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:40.363444 (+ 948us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:53:40.363451 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:53:40.363451 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:53:40.363456 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:53:40.363469 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:53:40.363473 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:53:40.363544 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 22:53:40.363608 (+ 64us) server_negotiation.cc:299] Negotiation successful
0806 22:53:40.363632 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 22:53:40.365355 15696 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42704:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:53:40.366947 15696 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:53:40.373514 16178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:40.369710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54212 (local address 127.28.218.194:33989)
0806 22:53:40.369878 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0806 22:53:40.369882 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:53:40.369894 (+ 12us) server_negotiation.cc:407] Connection header received
0806 22:53:40.369936 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:53:40.369938 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:53:40.369954 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:53:40.370211 (+ 257us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:53:40.370281 (+ 70us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:40.371105 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:40.372612 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:40.373106 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:40.373206 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:40.373302 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0806 22:53:40.373390 (+ 88us) server_negotiation.cc:299] Negotiation successful
0806 22:53:40.373426 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":94,"threads_started":1}
I20260806 22:53:40.373517 16174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:40.369357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.194:33989 (local address 127.0.0.1:54212)
0806 22:53:40.369638 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0806 22:53:40.369649 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:53:40.369703 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:53:40.370076 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:53:40.370077 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:53:40.370079 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:53:40.370133 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:53:40.370135 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:53:40.371229 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:53:40.371229 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:53:40.372502 (+ 1273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:53:40.372506 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:53:40.373205 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:53:40.373206 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:53:40.373284 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:40.373296 (+ 12us) client_negotiation.cc:769] Sending connection context
0806 22:53:40.373371 (+ 75us) client_negotiation.cc:240] Negotiation successful
0806 22:53:40.373425 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"mutex_wait_us":14,"thread_start_us":71,"threads_started":1}
I20260806 22:53:40.373811 16176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:40.369412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45664 (local address 127.28.218.193:34889)
0806 22:53:40.369637 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0806 22:53:40.369642 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:53:40.369654 (+ 12us) server_negotiation.cc:407] Connection header received
0806 22:53:40.369722 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:53:40.369724 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:53:40.369738 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:53:40.369844 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:53:40.370062 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:40.370974 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:40.372600 (+ 1626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:40.373312 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:40.373398 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:40.373489 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 22:53:40.373683 (+ 194us) server_negotiation.cc:299] Negotiation successful
0806 22:53:40.373727 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":102,"threads_started":1}
I20260806 22:53:40.374019 16175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:40.369320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.193:34889 (local address 127.0.0.1:45664)
0806 22:53:40.369497 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0806 22:53:40.369540 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 22:53:40.369637 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:53:40.369846 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:53:40.369850 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:53:40.369892 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:53:40.369961 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:53:40.369963 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:53:40.371066 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:53:40.371067 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:53:40.372504 (+ 1437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:53:40.372506 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:53:40.373530 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:53:40.373531 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:53:40.373592 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:40.373595 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:53:40.373907 (+ 312us) client_negotiation.cc:240] Negotiation successful
0806 22:53:40.373954 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"mutex_wait_us":54,"thread_start_us":54,"threads_started":1}
I20260806 22:53:40.374665 16177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:40.369540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.195:33461 (local address 127.0.0.1:44954)
0806 22:53:40.370270 (+ 730us) negotiation.cc:107] Waiting for socket to connect
0806 22:53:40.370282 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:53:40.370326 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:53:40.371347 (+ 1021us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:53:40.371348 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:53:40.371350 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:53:40.371401 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:53:40.371403 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:53:40.372262 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:53:40.372263 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:53:40.373398 (+ 1135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:53:40.373401 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:53:40.374418 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:53:40.374419 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:53:40.374484 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:40.374489 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:53:40.374537 (+ 48us) client_negotiation.cc:240] Negotiation successful
0806 22:53:40.374580 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":712,"mutex_wait_us":3,"thread_start_us":95,"threads_started":1}
I20260806 22:53:40.374622 15958 tablet_service.cc:1467] Processing CreateTablet for tablet 47d1b7510728490c8bd505e3529a8b60 (DEFAULT_TABLE table=test-table [id=2fa46c9b49364154a4f8421e1010ab94]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:53:40.374696 15821 tablet_service.cc:1467] Processing CreateTablet for tablet 47d1b7510728490c8bd505e3529a8b60 (DEFAULT_TABLE table=test-table [id=2fa46c9b49364154a4f8421e1010ab94]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:53:40.374953 16179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:40.370008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44954 (local address 127.28.218.195:33461)
0806 22:53:40.371111 (+ 1103us) server_negotiation.cc:206] Beginning negotiation
0806 22:53:40.371114 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:53:40.371182 (+ 68us) server_negotiation.cc:407] Connection header received
0806 22:53:40.371228 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:53:40.371230 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:53:40.371246 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:53:40.371349 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:53:40.371460 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:40.372166 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:40.373492 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:40.374231 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:40.374305 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:40.374382 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 22:53:40.374828 (+ 446us) server_negotiation.cc:299] Negotiation successful
0806 22:53:40.374878 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1070,"thread_start_us":95,"threads_started":1}
I20260806 22:53:40.375029 15958 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 47d1b7510728490c8bd505e3529a8b60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:53:40.375029 15821 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 47d1b7510728490c8bd505e3529a8b60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:53:40.375587 16097 tablet_service.cc:1467] Processing CreateTablet for tablet 47d1b7510728490c8bd505e3529a8b60 (DEFAULT_TABLE table=test-table [id=2fa46c9b49364154a4f8421e1010ab94]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:53:40.375839 16097 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 47d1b7510728490c8bd505e3529a8b60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:53:40.378451 16180 tablet_bootstrap.cc:492] T 47d1b7510728490c8bd505e3529a8b60 P f07b7eb6df6e478b844fd642f3473b63: Bootstrap starting.
I20260806 22:53:40.378856 16181 tablet_bootstrap.cc:492] T 47d1b7510728490c8bd505e3529a8b60 P 7e160ca4e8a848d09f702f6cb7659dc1: Bootstrap starting.
I20260806 22:53:40.379117 16180 tablet_bootstrap.cc:654] T 47d1b7510728490c8bd505e3529a8b60 P f07b7eb6df6e478b844fd642f3473b63: Neither blocks nor log segments found. Creating new log.
I20260806 22:53:40.379311 16182 tablet_bootstrap.cc:492] T 47d1b7510728490c8bd505e3529a8b60 P beb1dc1064cb40ac953e76c536fc9e8e: Bootstrap starting.
I20260806 22:53:40.379544 16180 log.cc:824] T 47d1b7510728490c8bd505e3529a8b60 P f07b7eb6df6e478b844fd642f3473b63: Log is configured to *not* fsync() on all Append() calls
I20260806 22:53:40.379567 16181 tablet_bootstrap.cc:654] T 47d1b7510728490c8bd505e3529a8b60 P 7e160ca4e8a848d09f702f6cb7659dc1: Neither blocks nor log segments found. Creating new log.
I20260806 22:53:40.379885 16181 log.cc:824] T 47d1b7510728490c8bd505e3529a8b60 P 7e160ca4e8a848d09f702f6cb7659dc1: Log is configured to *not* fsync() on all Append() calls
I20260806 22:53:40.379978 16182 tablet_bootstrap.cc:654] T 47d1b7510728490c8bd505e3529a8b60 P beb1dc1064cb40ac953e76c536fc9e8e: Neither blocks nor log segments found. Creating new log.
I20260806 22:53:40.380326 16182 log.cc:824] T 47d1b7510728490c8bd505e3529a8b60 P beb1dc1064cb40ac953e76c536fc9e8e: Log is configured to *not* fsync() on all Append() calls
I20260806 22:53:40.380340 16180 tablet_bootstrap.cc:492] T 47d1b7510728490c8bd505e3529a8b60 P f07b7eb6df6e478b844fd642f3473b63: No bootstrap required, opened a new log
I20260806 22:53:40.380439 16180 ts_tablet_manager.cc:1397] T 47d1b7510728490c8bd505e3529a8b60 P f07b7eb6df6e478b844fd642f3473b63: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:53:40.380810 16181 tablet_bootstrap.cc:492] T 47d1b7510728490c8bd505e3529a8b60 P 7e160ca4e8a848d09f702f6cb7659dc1: No bootstrap required, opened a new log
I20260806 22:53:40.380954 16181 ts_tablet_manager.cc:1397] T 47d1b7510728490c8bd505e3529a8b60 P 7e160ca4e8a848d09f702f6cb7659dc1: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260806 22:53:40.381093 16182 tablet_bootstrap.cc:492] T 47d1b7510728490c8bd505e3529a8b60 P beb1dc1064cb40ac953e76c536fc9e8e: No bootstrap required, opened a new log
I20260806 22:53:40.381172 16182 ts_tablet_manager.cc:1397] T 47d1b7510728490c8bd505e3529a8b60 P beb1dc1064cb40ac953e76c536fc9e8e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:53:40.382763 16181 raft_consensus.cc:348] T 47d1b7510728490c8bd505e3529a8b60 P 7e160ca4e8a848d09f702f6cb7659dc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "beb1dc1064cb40ac953e76c536fc9e8e" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 34889 } } peers { permanent_uuid: "7e160ca4e8a848d09f702f6cb7659dc1" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 33989 } } peers { permanent_uuid: "f07b7eb6df6e478b844fd642f3473b63" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 33461 } }
I20260806 22:53:40.382931 16181 raft_consensus.cc:374] T 47d1b7510728490c8bd505e3529a8b60 P 7e160ca4e8a848d09f702f6cb7659dc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:53:40.383008 16181 raft_consensus.cc:729] T 47d1b7510728490c8bd505e3529a8b60 P 7e160ca4e8a848d09f702f6cb7659dc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e160ca4e8a848d09f702f6cb7659dc1, State: Initialized, Role: FOLLOWER
I20260806 22:53:40.383164 16181 consensus_queue.cc:260] T 47d1b7510728490c8bd505e3529a8b60 P 7e160ca4e8a848d09f702f6cb7659dc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "beb1dc1064cb40ac953e76c536fc9e8e" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 34889 } } peers { permanent_uuid: "7e160ca4e8a848d09f702f6cb7659dc1" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 33989 } } peers { permanent_uuid: "f07b7eb6df6e478b844fd642f3473b63" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 33461 } }
I20260806 22:53:40.383195 16180 raft_consensus.cc:348] T 47d1b7510728490c8bd505e3529a8b60 P f07b7eb6df6e478b844fd642f3473b63 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "beb1dc1064cb40ac953e76c536fc9e8e" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 34889 } } peers { permanent_uuid: "7e160ca4e8a848d09f702f6cb7659dc1" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 33989 } } peers { permanent_uuid: "f07b7eb6df6e478b844fd642f3473b63" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 33461 } }
I20260806 22:53:40.383435 16180 raft_consensus.cc:374] T 47d1b7510728490c8bd505e3529a8b60 P f07b7eb6df6e478b844fd642f3473b63 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:53:40.383471 16181 ts_tablet_manager.cc:1428] T 47d1b7510728490c8bd505e3529a8b60 P 7e160ca4e8a848d09f702f6cb7659dc1: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:53:40.383500 16025 heartbeater.cc:503] Master 127.28.218.254:42877 was elected leader, sending a full tablet report...
I20260806 22:53:40.383543 16180 raft_consensus.cc:729] T 47d1b7510728490c8bd505e3529a8b60 P f07b7eb6df6e478b844fd642f3473b63 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f07b7eb6df6e478b844fd642f3473b63, State: Initialized, Role: FOLLOWER
I20260806 22:53:40.383664 16180 consensus_queue.cc:260] T 47d1b7510728490c8bd505e3529a8b60 P f07b7eb6df6e478b844fd642f3473b63 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "beb1dc1064cb40ac953e76c536fc9e8e" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 34889 } } peers { permanent_uuid: "7e160ca4e8a848d09f702f6cb7659dc1" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 33989 } } peers { permanent_uuid: "f07b7eb6df6e478b844fd642f3473b63" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 33461 } }
I20260806 22:53:40.383831 16182 raft_consensus.cc:348] T 47d1b7510728490c8bd505e3529a8b60 P beb1dc1064cb40ac953e76c536fc9e8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "beb1dc1064cb40ac953e76c536fc9e8e" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 34889 } } peers { permanent_uuid: "7e160ca4e8a848d09f702f6cb7659dc1" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 33989 } } peers { permanent_uuid: "f07b7eb6df6e478b844fd642f3473b63" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 33461 } }
I20260806 22:53:40.383999 16180 ts_tablet_manager.cc:1428] T 47d1b7510728490c8bd505e3529a8b60 P f07b7eb6df6e478b844fd642f3473b63: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260806 22:53:40.383999 16182 raft_consensus.cc:374] T 47d1b7510728490c8bd505e3529a8b60 P beb1dc1064cb40ac953e76c536fc9e8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:53:40.384042 16182 raft_consensus.cc:729] T 47d1b7510728490c8bd505e3529a8b60 P beb1dc1064cb40ac953e76c536fc9e8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: beb1dc1064cb40ac953e76c536fc9e8e, State: Initialized, Role: FOLLOWER
I20260806 22:53:40.384126 16165 heartbeater.cc:503] Master 127.28.218.254:42877 was elected leader, sending a full tablet report...
I20260806 22:53:40.384156 16182 consensus_queue.cc:260] T 47d1b7510728490c8bd505e3529a8b60 P beb1dc1064cb40ac953e76c536fc9e8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "beb1dc1064cb40ac953e76c536fc9e8e" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 34889 } } peers { permanent_uuid: "7e160ca4e8a848d09f702f6cb7659dc1" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 33989 } } peers { permanent_uuid: "f07b7eb6df6e478b844fd642f3473b63" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 33461 } }
I20260806 22:53:40.384388 16182 ts_tablet_manager.cc:1428] T 47d1b7510728490c8bd505e3529a8b60 P beb1dc1064cb40ac953e76c536fc9e8e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 22:53:40.384471 15889 heartbeater.cc:503] Master 127.28.218.254:42877 was elected leader, sending a full tablet report...
W20260806 22:53:40.456072 16026 tablet.cc:2369] T 47d1b7510728490c8bd505e3529a8b60 P 7e160ca4e8a848d09f702f6cb7659dc1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:53:40.543648 16189 raft_consensus.cc:482] T 47d1b7510728490c8bd505e3529a8b60 P f07b7eb6df6e478b844fd642f3473b63 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:53:40.543782 16189 raft_consensus.cc:504] T 47d1b7510728490c8bd505e3529a8b60 P f07b7eb6df6e478b844fd642f3473b63 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "beb1dc1064cb40ac953e76c536fc9e8e" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 34889 } } peers { permanent_uuid: "7e160ca4e8a848d09f702f6cb7659dc1" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 33989 } } peers { permanent_uuid: "f07b7eb6df6e478b844fd642f3473b63" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 33461 } }
I20260806 22:53:40.544262 16189 leader_election.cc:288] T 47d1b7510728490c8bd505e3529a8b60 P f07b7eb6df6e478b844fd642f3473b63 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers beb1dc1064cb40ac953e76c536fc9e8e (127.28.218.193:34889), 7e160ca4e8a848d09f702f6cb7659dc1 (127.28.218.194:33989)
I20260806 22:53:40.545210 16191 raft_consensus.cc:482] T 47d1b7510728490c8bd505e3529a8b60 P beb1dc1064cb40ac953e76c536fc9e8e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:53:40.545323 16191 raft_consensus.cc:504] T 47d1b7510728490c8bd505e3529a8b60 P beb1dc1064cb40ac953e76c536fc9e8e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "beb1dc1064cb40ac953e76c536fc9e8e" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 34889 } } peers { permanent_uuid: "7e160ca4e8a848d09f702f6cb7659dc1" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 33989 } } peers { permanent_uuid: "f07b7eb6df6e478b844fd642f3473b63" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 33461 } }
I20260806 22:53:40.545642 16191 leader_election.cc:288] T 47d1b7510728490c8bd505e3529a8b60 P beb1dc1064cb40ac953e76c536fc9e8e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e160ca4e8a848d09f702f6cb7659dc1 (127.28.218.194:33989), f07b7eb6df6e478b844fd642f3473b63 (127.28.218.195:33461)
I20260806 22:53:40.548190 16167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:40.544454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.194:33989 (local address 127.28.218.195:59585)
0806 22:53:40.544520 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0806 22:53:40.544534 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 22:53:40.544585 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:53:40.544713 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:53:40.544714 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:53:40.544715 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:53:40.544791 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:53:40.544792 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:53:40.546002 (+ 1210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:53:40.546003 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:53:40.547238 (+ 1235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:53:40.547241 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:53:40.547954 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:53:40.547955 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:53:40.548018 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:40.548022 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:53:40.548082 (+ 60us) client_negotiation.cc:240] Negotiation successful
0806 22:53:40.548120 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44,"mutex_wait_us":26}
I20260806 22:53:40.548213 16178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:40.544540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.195:59585 (local address 127.28.218.194:33989)
0806 22:53:40.544594 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0806 22:53:40.544596 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:53:40.544624 (+ 28us) server_negotiation.cc:407] Connection header received
0806 22:53:40.544652 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:53:40.544653 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:53:40.544664 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:53:40.544711 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:53:40.544915 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:40.545640 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:40.547328 (+ 1688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:40.547859 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:40.547930 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:40.547937 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0806 22:53:40.548111 (+ 174us) server_negotiation.cc:299] Negotiation successful
0806 22:53:40.548150 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 22:53:40.548533 15978 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47d1b7510728490c8bd505e3529a8b60" candidate_uuid: "f07b7eb6df6e478b844fd642f3473b63" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e160ca4e8a848d09f702f6cb7659dc1" is_pre_election: true
I20260806 22:53:40.548687 15978 raft_consensus.cc:2454] T 47d1b7510728490c8bd505e3529a8b60 P 7e160ca4e8a848d09f702f6cb7659dc1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f07b7eb6df6e478b844fd642f3473b63 in term 0.
I20260806 22:53:40.548843 16176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:40.544519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.195:37409 (local address 127.28.218.193:34889)
0806 22:53:40.544571 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0806 22:53:40.544573 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:53:40.544726 (+ 153us) server_negotiation.cc:407] Connection header received
0806 22:53:40.544797 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:53:40.544798 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:53:40.544811 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:53:40.544862 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:53:40.545060 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:40.545747 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:40.547459 (+ 1712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:40.548032 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:40.548122 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:40.548129 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0806 22:53:40.548734 (+ 605us) server_negotiation.cc:299] Negotiation successful
0806 22:53:40.548769 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":39,"server-negotiator.queue_time_us":27}
I20260806 22:53:40.548923 16047 leader_election.cc:302] T 47d1b7510728490c8bd505e3529a8b60 P f07b7eb6df6e478b844fd642f3473b63 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e160ca4e8a848d09f702f6cb7659dc1, f07b7eb6df6e478b844fd642f3473b63; no voters:
I20260806 22:53:40.549041 16189 raft_consensus.cc:2781] T 47d1b7510728490c8bd505e3529a8b60 P f07b7eb6df6e478b844fd642f3473b63 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:53:40.549086 16189 raft_consensus.cc:482] T 47d1b7510728490c8bd505e3529a8b60 P f07b7eb6df6e478b844fd642f3473b63 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:53:40.549108 16189 raft_consensus.cc:3037] T 47d1b7510728490c8bd505e3529a8b60 P f07b7eb6df6e478b844fd642f3473b63 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:53:40.549337 16192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:40.544472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.193:34889 (local address 127.28.218.195:37409)
0806 22:53:40.544640 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0806 22:53:40.544655 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 22:53:40.544740 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:53:40.544884 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:53:40.544885 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:53:40.544887 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:53:40.544954 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:53:40.544956 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:53:40.545848 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:53:40.545849 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:53:40.547381 (+ 1532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:53:40.547384 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:53:40.548528 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:53:40.548529 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:53:40.548598 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:40.548601 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:53:40.549205 (+ 604us) client_negotiation.cc:240] Negotiation successful
0806 22:53:40.549255 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"mutex_wait_us":1,"thread_start_us":113,"threads_started":1}
I20260806 22:53:40.549475 15891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:40.545777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.194:33989 (local address 127.28.218.193:58073)
0806 22:53:40.545849 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0806 22:53:40.545860 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:53:40.545899 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:53:40.546332 (+ 433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:53:40.546333 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:53:40.546335 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:53:40.546391 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:53:40.546392 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:53:40.547309 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:53:40.547310 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:53:40.548264 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:53:40.548268 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:53:40.549271 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:53:40.549272 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:53:40.549344 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:40.549347 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:53:40.549378 (+ 31us) client_negotiation.cc:240] Negotiation successful
0806 22:53:40.549413 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260806 22:53:40.549744 16194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:40.546031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.193:58073 (local address 127.28.218.194:33989)
0806 22:53:40.546193 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0806 22:53:40.546195 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:53:40.546202 (+ 7us) server_negotiation.cc:407] Connection header received
0806 22:53:40.546221 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:53:40.546221 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:53:40.546229 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:53:40.546300 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:53:40.546480 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:40.547216 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:40.548363 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:40.549159 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:40.549605 (+ 446us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:40.549613 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 22:53:40.549632 (+ 19us) server_negotiation.cc:299] Negotiation successful
0806 22:53:40.549672 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":98,"threads_started":1}
I20260806 22:53:40.550123 16189 raft_consensus.cc:504] T 47d1b7510728490c8bd505e3529a8b60 P f07b7eb6df6e478b844fd642f3473b63 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "beb1dc1064cb40ac953e76c536fc9e8e" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 34889 } } peers { permanent_uuid: "7e160ca4e8a848d09f702f6cb7659dc1" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 33989 } } peers { permanent_uuid: "f07b7eb6df6e478b844fd642f3473b63" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 33461 } }
I20260806 22:53:40.550271 16189 leader_election.cc:288] T 47d1b7510728490c8bd505e3529a8b60 P f07b7eb6df6e478b844fd642f3473b63 [CANDIDATE]: Term 1 election: Requested vote from peers beb1dc1064cb40ac953e76c536fc9e8e (127.28.218.193:34889), 7e160ca4e8a848d09f702f6cb7659dc1 (127.28.218.194:33989)
I20260806 22:53:40.550280 16193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:40.545835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.195:33461 (local address 127.28.218.193:39793)
0806 22:53:40.546398 (+ 563us) negotiation.cc:107] Waiting for socket to connect
0806 22:53:40.546407 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:53:40.546475 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:53:40.546621 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:53:40.546622 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:53:40.546624 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:53:40.546673 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:53:40.546674 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:53:40.547449 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:53:40.547450 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:53:40.549186 (+ 1736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:53:40.549189 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:53:40.549994 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:53:40.549995 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:53:40.550064 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:40.550067 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:53:40.550166 (+ 99us) client_negotiation.cc:240] Negotiation successful
0806 22:53:40.550202 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":545,"mutex_wait_us":28,"thread_start_us":85,"threads_started":1}
I20260806 22:53:40.550273 16179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:40.546053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.193:39793 (local address 127.28.218.195:33461)
0806 22:53:40.546093 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 22:53:40.546094 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:53:40.546452 (+ 358us) server_negotiation.cc:407] Connection header received
0806 22:53:40.546539 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:53:40.546539 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:53:40.546547 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:53:40.546572 (+ 25us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:53:40.546734 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:40.547364 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:40.549279 (+ 1915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:40.549827 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:40.549876 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:40.549885 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 22:53:40.550171 (+ 286us) server_negotiation.cc:299] Negotiation successful
0806 22:53:40.550203 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 22:53:40.550542 15978 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47d1b7510728490c8bd505e3529a8b60" candidate_uuid: "beb1dc1064cb40ac953e76c536fc9e8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e160ca4e8a848d09f702f6cb7659dc1" is_pre_election: true
I20260806 22:53:40.550626 15978 raft_consensus.cc:2454] T 47d1b7510728490c8bd505e3529a8b60 P 7e160ca4e8a848d09f702f6cb7659dc1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate beb1dc1064cb40ac953e76c536fc9e8e in term 0.
I20260806 22:53:40.550652 15842 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47d1b7510728490c8bd505e3529a8b60" candidate_uuid: "f07b7eb6df6e478b844fd642f3473b63" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "beb1dc1064cb40ac953e76c536fc9e8e"
I20260806 22:53:40.550680 15843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47d1b7510728490c8bd505e3529a8b60" candidate_uuid: "f07b7eb6df6e478b844fd642f3473b63" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "beb1dc1064cb40ac953e76c536fc9e8e" is_pre_election: true
I20260806 22:53:40.550751 15842 raft_consensus.cc:3037] T 47d1b7510728490c8bd505e3529a8b60 P beb1dc1064cb40ac953e76c536fc9e8e [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:53:40.550933 15776 leader_election.cc:302] T 47d1b7510728490c8bd505e3529a8b60 P beb1dc1064cb40ac953e76c536fc9e8e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e160ca4e8a848d09f702f6cb7659dc1, beb1dc1064cb40ac953e76c536fc9e8e; no voters:
I20260806 22:53:40.550984 16117 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47d1b7510728490c8bd505e3529a8b60" candidate_uuid: "beb1dc1064cb40ac953e76c536fc9e8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f07b7eb6df6e478b844fd642f3473b63" is_pre_election: true
I20260806 22:53:40.551105 16117 raft_consensus.cc:2379] T 47d1b7510728490c8bd505e3529a8b60 P f07b7eb6df6e478b844fd642f3473b63 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate beb1dc1064cb40ac953e76c536fc9e8e in current term 1: Already voted for candidate f07b7eb6df6e478b844fd642f3473b63 in this term.
I20260806 22:53:40.551321 15978 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47d1b7510728490c8bd505e3529a8b60" candidate_uuid: "f07b7eb6df6e478b844fd642f3473b63" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e160ca4e8a848d09f702f6cb7659dc1"
I20260806 22:53:40.551390 15978 raft_consensus.cc:3037] T 47d1b7510728490c8bd505e3529a8b60 P 7e160ca4e8a848d09f702f6cb7659dc1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:53:40.551699 15842 raft_consensus.cc:2454] T 47d1b7510728490c8bd505e3529a8b60 P beb1dc1064cb40ac953e76c536fc9e8e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f07b7eb6df6e478b844fd642f3473b63 in term 1.
I20260806 22:53:40.551795 16191 raft_consensus.cc:2741] T 47d1b7510728490c8bd505e3529a8b60 P beb1dc1064cb40ac953e76c536fc9e8e [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260806 22:53:40.551899 16046 leader_election.cc:302] T 47d1b7510728490c8bd505e3529a8b60 P f07b7eb6df6e478b844fd642f3473b63 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: beb1dc1064cb40ac953e76c536fc9e8e, f07b7eb6df6e478b844fd642f3473b63; no voters:
I20260806 22:53:40.551966 16189 raft_consensus.cc:2781] T 47d1b7510728490c8bd505e3529a8b60 P f07b7eb6df6e478b844fd642f3473b63 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:53:40.552109 16189 raft_consensus.cc:686] T 47d1b7510728490c8bd505e3529a8b60 P f07b7eb6df6e478b844fd642f3473b63 [term 1 LEADER]: Becoming Leader. State: Replica: f07b7eb6df6e478b844fd642f3473b63, State: Running, Role: LEADER
I20260806 22:53:40.552196 16189 consensus_queue.cc:237] T 47d1b7510728490c8bd505e3529a8b60 P f07b7eb6df6e478b844fd642f3473b63 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "beb1dc1064cb40ac953e76c536fc9e8e" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 34889 } } peers { permanent_uuid: "7e160ca4e8a848d09f702f6cb7659dc1" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 33989 } } peers { permanent_uuid: "f07b7eb6df6e478b844fd642f3473b63" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 33461 } }
I20260806 22:53:40.552302 15978 raft_consensus.cc:2454] T 47d1b7510728490c8bd505e3529a8b60 P 7e160ca4e8a848d09f702f6cb7659dc1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f07b7eb6df6e478b844fd642f3473b63 in term 1.
I20260806 22:53:40.553009 15695 catalog_manager.cc:5495] T 47d1b7510728490c8bd505e3529a8b60 P f07b7eb6df6e478b844fd642f3473b63 reported cstate change: term changed from 0 to 1, leader changed from <none> to f07b7eb6df6e478b844fd642f3473b63 (127.28.218.195). New cstate: current_term: 1 leader_uuid: "f07b7eb6df6e478b844fd642f3473b63" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "beb1dc1064cb40ac953e76c536fc9e8e" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e160ca4e8a848d09f702f6cb7659dc1" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 33989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f07b7eb6df6e478b844fd642f3473b63" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 33461 } health_report { overall_health: HEALTHY } } }
W20260806 22:53:40.574182 15890 tablet.cc:2369] T 47d1b7510728490c8bd505e3529a8b60 P beb1dc1064cb40ac953e76c536fc9e8e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 22:53:40.595327 16166 tablet.cc:2369] T 47d1b7510728490c8bd505e3529a8b60 P f07b7eb6df6e478b844fd642f3473b63: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:53:40.600855 16179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:40.598423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44960 (local address 127.28.218.195:33461)
0806 22:53:40.598488 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0806 22:53:40.598490 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:53:40.598498 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:53:40.598535 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:53:40.598536 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:53:40.598552 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:53:40.598610 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:53:40.598693 (+ 83us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:40.599266 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:40.600121 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:40.600295 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:40.600346 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:40.600509 (+ 163us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:53:40.600602 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:53:40.600663 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 22:53:40.600711 (+ 48us) server_negotiation.cc:299] Negotiation successful
0806 22:53:40.600728 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 22:53:40.602903 15978 raft_consensus.cc:1261] T 47d1b7510728490c8bd505e3529a8b60 P 7e160ca4e8a848d09f702f6cb7659dc1 [term 1 FOLLOWER]: Refusing update from remote peer f07b7eb6df6e478b844fd642f3473b63: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:53:40.602916 15842 raft_consensus.cc:1261] T 47d1b7510728490c8bd505e3529a8b60 P beb1dc1064cb40ac953e76c536fc9e8e [term 1 FOLLOWER]: Refusing update from remote peer f07b7eb6df6e478b844fd642f3473b63: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:53:40.603242 16195 consensus_queue.cc:1035] T 47d1b7510728490c8bd505e3529a8b60 P f07b7eb6df6e478b844fd642f3473b63 [LEADER]: Connected to new peer: Peer: permanent_uuid: "beb1dc1064cb40ac953e76c536fc9e8e" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:53:40.603369 16189 consensus_queue.cc:1035] T 47d1b7510728490c8bd505e3529a8b60 P f07b7eb6df6e478b844fd642f3473b63 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e160ca4e8a848d09f702f6cb7659dc1" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 33989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:53:40.608053 15696 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:42704:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:53:40.608163 15696 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:42704:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:53:40.608946 15696 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bd1d36cf82604aabb3df0becaaab2b3d: Sending DeleteTablet for 3 replicas of tablet 47d1b7510728490c8bd505e3529a8b60
I20260806 22:53:40.609366 15958 tablet_service.cc:1514] Processing DeleteTablet for tablet 47d1b7510728490c8bd505e3529a8b60 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:53:40 UTC) from {username='slave'} at 127.0.0.1:54212
I20260806 22:53:40.609395 15821 tablet_service.cc:1514] Processing DeleteTablet for tablet 47d1b7510728490c8bd505e3529a8b60 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:53:40 UTC) from {username='slave'} at 127.0.0.1:45664
I20260806 22:53:40.609634 16209 tablet_replica.cc:330] T 47d1b7510728490c8bd505e3529a8b60 P 7e160ca4e8a848d09f702f6cb7659dc1: stopping tablet replica
I20260806 22:53:40.609645 16097 tablet_service.cc:1514] Processing DeleteTablet for tablet 47d1b7510728490c8bd505e3529a8b60 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:53:40 UTC) from {username='slave'} at 127.0.0.1:44954
I20260806 22:53:40.609750 16209 raft_consensus.cc:2229] T 47d1b7510728490c8bd505e3529a8b60 P 7e160ca4e8a848d09f702f6cb7659dc1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:53:40.609867 16209 raft_consensus.cc:2258] T 47d1b7510728490c8bd505e3529a8b60 P 7e160ca4e8a848d09f702f6cb7659dc1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:53:40.610044 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 15758
I20260806 22:53:40.609887 16211 tablet_replica.cc:330] T 47d1b7510728490c8bd505e3529a8b60 P f07b7eb6df6e478b844fd642f3473b63: stopping tablet replica
I20260806 22:53:40.610827 16211 raft_consensus.cc:2229] T 47d1b7510728490c8bd505e3529a8b60 P f07b7eb6df6e478b844fd642f3473b63 [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:53:40.611037 16211 raft_consensus.cc:2258] T 47d1b7510728490c8bd505e3529a8b60 P f07b7eb6df6e478b844fd642f3473b63 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:53:40.611610 16211 ts_tablet_manager.cc:1905] T 47d1b7510728490c8bd505e3529a8b60 P f07b7eb6df6e478b844fd642f3473b63: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:53:40.613729 16211 ts_tablet_manager.cc:1918] T 47d1b7510728490c8bd505e3529a8b60 P f07b7eb6df6e478b844fd642f3473b63: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:53:40.613806 16211 log.cc:1196] T 47d1b7510728490c8bd505e3529a8b60 P f07b7eb6df6e478b844fd642f3473b63: Deleting WAL directory at /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056586788339-29547-0/minicluster-data/ts-2/wal/wals/47d1b7510728490c8bd505e3529a8b60
I20260806 22:53:40.614070 16211 ts_tablet_manager.cc:1939] T 47d1b7510728490c8bd505e3529a8b60 P f07b7eb6df6e478b844fd642f3473b63: Deleting consensus metadata
I20260806 22:53:40.614507 15680 catalog_manager.cc:4841] TS f07b7eb6df6e478b844fd642f3473b63 (127.28.218.195:33461): tablet 47d1b7510728490c8bd505e3529a8b60 (table test-table [id=2fa46c9b49364154a4f8421e1010ab94]) successfully deleted
I20260806 22:53:40.618194 16209 ts_tablet_manager.cc:1905] T 47d1b7510728490c8bd505e3529a8b60 P 7e160ca4e8a848d09f702f6cb7659dc1: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260806 22:53:40.618201 15680 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.28.218.193:34889 (error 108)
I20260806 22:53:40.618472 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 15892
I20260806 22:53:40.618685 16177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:40.618571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.193:34889 (local address 127.0.0.1:45668)
0806 22:53:40.618617 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0806 22:53:40.618651 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.193:34889: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260806 22:53:40.619643 15680 catalog_manager.cc:4584] TS beb1dc1064cb40ac953e76c536fc9e8e (127.28.218.193:34889): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 47d1b7510728490c8bd505e3529a8b60: Network error: Client connection negotiation failed: client connection to 127.28.218.193:34889: connect: Connection refused (error 111)
I20260806 22:53:40.627955 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 16028
I20260806 22:53:40.628189 16177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:40.627995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.194:33989 (local address 127.0.0.1:54224)
0806 22:53:40.628111 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0806 22:53:40.628140 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.194:33989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
I20260806 22:53:40.633129 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 15664
2026-08-06T22:53:40Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (909 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056586788339-29547-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:53:40 dist-test-slave-1xrh krb5kdc[16215](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:53:40 dist-test-slave-1xrh krb5kdc[16215](info): set up 2 sockets
Aug 06 22:53:40 dist-test-slave-1xrh krb5kdc[16215](info): commencing operation
krb5kdc: starting...
W20260806 22:53:41.687533 29547 mini_kdc.cc:121] Time spent starting KDC: real 1.034s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056586788339-29547-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056586788339-29547-0/krb5kdc/test-user.keytab.
Aug 06 22:53:41 dist-test-slave-1xrh krb5kdc[16215](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056821, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:53:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:53:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.218.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.218.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.218.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab.
Entry for principal kudu/127.28.218.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab.
I20260806 22:53:41.810969 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056586788339-29547-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056586788339-29547-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056586788339-29547-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.218.254:43035
--webserver_interface=127.28.218.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab
--principal=kudu/127.28.218.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.218.212:38639
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.218.254:43035
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:53:41.891992 16228 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:53:41.892233 16228 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:53:41.892271 16228 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:53:41.893355 16228 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:53:41.893428 16228 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:53:41.893455 16228 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:53:41.893484 16228 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:53:41.893512 16228 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:53:41.895037 16228 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:38639
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056586788339-29547-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056586788339-29547-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.218.254:43035
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab
--principal=kudu/127.28.218.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.218.254:43035
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.218.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16228
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056586788339-29547-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:53:41.895334 16228 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:53:41.895555 16228 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:53:41.900285 16233 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:53:41.900300 16235 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:53:41.900416 16237 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:53:41.900631 16228 server_base.cc:1034] running on GCE node
I20260806 22:53:41.900830 16228 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:53:41.901084 16228 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:53:41.902228 16228 hybrid_clock.cc:648] HybridClock initialized: now 1786056821902210 us; error 33 us; skew 500 ppm
Aug 06 22:53:41 dist-test-slave-1xrh krb5kdc[16215](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056821, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:53:41.909039 16228 init.cc:377] Logged in from keytab as kudu/127.28.218.254@KRBTEST.COM (short username kudu)
I20260806 22:53:41.909524 16228 webserver.cc:460] Webserver started at http://127.28.218.254:38357/ using document root <none> and password file <none>
I20260806 22:53:41.909754 16228 fs_manager.cc:359] Metadata directory not provided
I20260806 22:53:41.909799 16228 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:53:41.909948 16228 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:53:41.910964 16228 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056586788339-29547-0/minicluster-data/master-0/data/instance:
uuid: "5757ffaa36f54563b9caf32ba61be811"
format_stamp: "Formatted at 2026-08-06 22:53:41 on dist-test-slave-1xrh"
server_key: "670848ff218bf29e41fa502707051f39"
server_key_iv: "17910f36f32b21211a878fc75d3d409e"
server_key_version: "encryptionkey@0"
I20260806 22:53:41.911438 16228 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056586788339-29547-0/minicluster-data/master-0/wal/instance:
uuid: "5757ffaa36f54563b9caf32ba61be811"
format_stamp: "Formatted at 2026-08-06 22:53:41 on dist-test-slave-1xrh"
server_key: "670848ff218bf29e41fa502707051f39"
server_key_iv: "17910f36f32b21211a878fc75d3d409e"
server_key_version: "encryptionkey@0"
I20260806 22:53:41.912788 16228 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:53:41.913534 16244 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:53:41.913727 16228 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:53:41.913832 16228 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056586788339-29547-0/minicluster-data/master-0/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056586788339-29547-0/minicluster-data/master-0/wal
uuid: "5757ffaa36f54563b9caf32ba61be811"
format_stamp: "Formatted at 2026-08-06 22:53:41 on dist-test-slave-1xrh"
server_key: "670848ff218bf29e41fa502707051f39"
server_key_iv: "17910f36f32b21211a878fc75d3d409e"
server_key_version: "encryptionkey@0"
I20260806 22:53:41.913919 16228 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056586788339-29547-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056586788339-29547-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056586788339-29547-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:53:41.927099 16228 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:53:41.929610 16228 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:53:41.929808 16228 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:53:41.934072 16228 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.254:43035
I20260806 22:53:41.934108 16299 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.254:43035 every 8 connection(s)
I20260806 22:53:41.934454 16228 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
I20260806 22:53:41.935313 16300 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:53:41.935853 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 16228
I20260806 22:53:41.935947 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056586788339-29547-0/minicluster-data/master-0/wal/instance
I20260806 22:53:41.936187 29547 external_mini_cluster.cc:1428] Setting key 4d2262d50ba1d8b46bd07a0d2d2f3513
I20260806 22:53:41.938429 16300 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5757ffaa36f54563b9caf32ba61be811: Bootstrap starting.
I20260806 22:53:41.939154 16300 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5757ffaa36f54563b9caf32ba61be811: Neither blocks nor log segments found. Creating new log.
I20260806 22:53:41.939507 16300 log.cc:824] T 00000000000000000000000000000000 P 5757ffaa36f54563b9caf32ba61be811: Log is configured to *not* fsync() on all Append() calls
I20260806 22:53:41.940183 16300 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5757ffaa36f54563b9caf32ba61be811: No bootstrap required, opened a new log
I20260806 22:53:41.942147 16300 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5757ffaa36f54563b9caf32ba61be811 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5757ffaa36f54563b9caf32ba61be811" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 43035 } }
I20260806 22:53:41.942276 16300 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5757ffaa36f54563b9caf32ba61be811 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:53:41.942298 16300 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5757ffaa36f54563b9caf32ba61be811 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5757ffaa36f54563b9caf32ba61be811, State: Initialized, Role: FOLLOWER
I20260806 22:53:41.942436 16300 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5757ffaa36f54563b9caf32ba61be811 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5757ffaa36f54563b9caf32ba61be811" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 43035 } }
I20260806 22:53:41.942543 16300 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5757ffaa36f54563b9caf32ba61be811 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:53:41.942567 16300 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5757ffaa36f54563b9caf32ba61be811 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:53:41.942637 16300 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5757ffaa36f54563b9caf32ba61be811 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:53:41.943212 16300 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5757ffaa36f54563b9caf32ba61be811 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5757ffaa36f54563b9caf32ba61be811" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 43035 } }
Aug 06 22:53:41 dist-test-slave-1xrh krb5kdc[16215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056821, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.218.254@KRBTEST.COM
I20260806 22:53:41.943328 16300 leader_election.cc:302] T 00000000000000000000000000000000 P 5757ffaa36f54563b9caf32ba61be811 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5757ffaa36f54563b9caf32ba61be811; no voters:
I20260806 22:53:41.943498 16300 leader_election.cc:288] T 00000000000000000000000000000000 P 5757ffaa36f54563b9caf32ba61be811 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:53:41.943564 16305 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5757ffaa36f54563b9caf32ba61be811 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:53:41.943722 16305 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5757ffaa36f54563b9caf32ba61be811 [term 1 LEADER]: Becoming Leader. State: Replica: 5757ffaa36f54563b9caf32ba61be811, State: Running, Role: LEADER
I20260806 22:53:41.943763 16300 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5757ffaa36f54563b9caf32ba61be811 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:53:41.943935 16305 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5757ffaa36f54563b9caf32ba61be811 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5757ffaa36f54563b9caf32ba61be811" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 43035 } }
I20260806 22:53:41.944321 16306 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5757ffaa36f54563b9caf32ba61be811 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5757ffaa36f54563b9caf32ba61be811" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5757ffaa36f54563b9caf32ba61be811" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 43035 } } }
I20260806 22:53:41.944408 16306 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5757ffaa36f54563b9caf32ba61be811 [sys.catalog]: This master's current role is: LEADER
I20260806 22:53:41.944551 16307 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5757ffaa36f54563b9caf32ba61be811 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5757ffaa36f54563b9caf32ba61be811. Latest consensus state: current_term: 1 leader_uuid: "5757ffaa36f54563b9caf32ba61be811" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5757ffaa36f54563b9caf32ba61be811" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 43035 } } }
I20260806 22:53:41.944662 16307 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5757ffaa36f54563b9caf32ba61be811 [sys.catalog]: This master's current role is: LEADER
I20260806 22:53:41.945497 16311 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:53:41.945987 16311 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:53:41.947893 16303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:41.936770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45458 (local address 127.28.218.254:43035)
0806 22:53:41.936953 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0806 22:53:41.936962 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:53:41.936996 (+ 34us) server_negotiation.cc:407] Connection header received
0806 22:53:41.937521 (+ 525us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:53:41.937530 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:53:41.937589 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:53:41.937741 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:53:41.938378 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:41.939804 (+ 1426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:41.940664 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:41.941087 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:41.941215 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:41.943921 (+ 2706us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:53:41.943925 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:53:41.943925 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:53:41.943938 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:53:41.946006 (+ 2068us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:53:41.946464 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:53:41.946468 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:53:41.946468 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:53:41.946619 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:53:41.946841 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:53:41.946841 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:53:41.946842 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:53:41.947174 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:53:41.947334 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0806 22:53:41.947531 (+ 197us) server_negotiation.cc:299] Negotiation successful
0806 22:53:41.947689 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":77,"threads_started":1}
I20260806 22:53:41.947921 16311 catalog_manager.cc:1347] Generated new cluster ID: 01fd6840a5884d8ba4b7e18f21be747c
I20260806 22:53:41.947964 16311 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:53:41.960031 16311 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:53:41.960541 16311 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:53:41.964332 16311 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5757ffaa36f54563b9caf32ba61be811: Generated new TSK 0
I20260806 22:53:41.964497 16311 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.28.218.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.218.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.218.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.193.keytab.
Entry for principal kudu/127.28.218.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.193.keytab.
I20260806 22:53:41.994624 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056586788339-29547-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056586788339-29547-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056586788339-29547-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056586788339-29547-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.218.193:0
--local_ip_for_outbound_sockets=127.28.218.193
--webserver_interface=127.28.218.193
--webserver_port=0
--tserver_master_addrs=127.28.218.254:43035
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.193.keytab
--principal=kudu/127.28.218.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.218.212:38639
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:53:42.075706 16327 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:53:42.075899 16327 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:53:42.075922 16327 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:53:42.077004 16327 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:53:42.077037 16327 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:53:42.077062 16327 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.218.193
I20260806 22:53:42.078402 16327 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:38639
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056586788339-29547-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056586788339-29547-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.193.keytab
--principal=kudu/127.28.218.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.218.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056586788339-29547-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.218.193
--webserver_port=0
--tserver_master_addrs=127.28.218.254:43035
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16327
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.218.193
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056586788339-29547-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:53:42.078615 16327 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:53:42.078836 16327 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:53:42.083936 16334 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:53:42.083928 16337 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:53:42.083927 16333 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:53:42.084028 16327 server_base.cc:1034] running on GCE node
I20260806 22:53:42.084427 16327 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:53:42.084620 16327 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:53:42.085754 16327 hybrid_clock.cc:648] HybridClock initialized: now 1786056822085745 us; error 27 us; skew 500 ppm
Aug 06 22:53:42 dist-test-slave-1xrh krb5kdc[16215](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056822, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:53:42.088429 16327 init.cc:377] Logged in from keytab as kudu/127.28.218.193@KRBTEST.COM (short username kudu)
I20260806 22:53:42.088770 16327 webserver.cc:460] Webserver started at http://127.28.218.193:43067/ using document root <none> and password file <none>
I20260806 22:53:42.089039 16327 fs_manager.cc:359] Metadata directory not provided
I20260806 22:53:42.089138 16327 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:53:42.089294 16327 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:53:42.090287 16327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056586788339-29547-0/minicluster-data/ts-0/data/instance:
uuid: "2515e82235204a06a58e63d17fc1df96"
format_stamp: "Formatted at 2026-08-06 22:53:42 on dist-test-slave-1xrh"
server_key: "c18b79f1edbc5a01b3438c7ba3393e52"
server_key_iv: "439e84e61e60536a0808dd3902b6d33f"
server_key_version: "encryptionkey@0"
I20260806 22:53:42.090727 16327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056586788339-29547-0/minicluster-data/ts-0/wal/instance:
uuid: "2515e82235204a06a58e63d17fc1df96"
format_stamp: "Formatted at 2026-08-06 22:53:42 on dist-test-slave-1xrh"
server_key: "c18b79f1edbc5a01b3438c7ba3393e52"
server_key_iv: "439e84e61e60536a0808dd3902b6d33f"
server_key_version: "encryptionkey@0"
I20260806 22:53:42.092120 16327 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 22:53:42.092841 16346 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:53:42.093007 16327 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 22:53:42.093109 16327 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056586788339-29547-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056586788339-29547-0/minicluster-data/ts-0/wal
uuid: "2515e82235204a06a58e63d17fc1df96"
format_stamp: "Formatted at 2026-08-06 22:53:42 on dist-test-slave-1xrh"
server_key: "c18b79f1edbc5a01b3438c7ba3393e52"
server_key_iv: "439e84e61e60536a0808dd3902b6d33f"
server_key_version: "encryptionkey@0"
I20260806 22:53:42.093192 16327 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056586788339-29547-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056586788339-29547-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056586788339-29547-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:53:42.103533 16327 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:53:42.105764 16327 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:53:42.105937 16327 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:53:42.106154 16327 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:53:42.106457 16327 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:53:42.106515 16327 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:53:42.106570 16327 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:53:42.106598 16327 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:53:42.112527 16327 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.193:38227
I20260806 22:53:42.112624 16462 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.193:38227 every 8 connection(s)
I20260806 22:53:42.112900 16327 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056586788339-29547-0/minicluster-data/ts-0/data/info.pb
Aug 06 22:53:42 dist-test-slave-1xrh krb5kdc[16215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056822, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.193@KRBTEST.COM for kudu/127.28.218.254@KRBTEST.COM
I20260806 22:53:42.119486 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 16327
I20260806 22:53:42.119611 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056586788339-29547-0/minicluster-data/ts-0/wal/instance
I20260806 22:53:42.119880 29547 external_mini_cluster.cc:1428] Setting key eba153dbc796702b9969a65189131478
I20260806 22:53:42.122781 16303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:42.113657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.193:57575 (local address 127.28.218.254:43035)
0806 22:53:42.113751 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0806 22:53:42.113753 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:53:42.113933 (+ 180us) server_negotiation.cc:407] Connection header received
0806 22:53:42.114469 (+ 536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:53:42.114470 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:53:42.114483 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:53:42.114552 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:53:42.115578 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:42.116100 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:42.117168 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:42.117390 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:42.117494 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:42.119523 (+ 2029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:53:42.119524 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:53:42.119525 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:53:42.119533 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:53:42.121407 (+ 1874us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:53:42.121790 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:53:42.121791 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:53:42.121792 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:53:42.121856 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:53:42.122099 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:53:42.122100 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:53:42.122100 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:53:42.122257 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:53:42.122324 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 22:53:42.122595 (+ 271us) server_negotiation.cc:299] Negotiation successful
0806 22:53:42.122689 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 22:53:42.122951 16466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:42.113614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:43035 (local address 127.28.218.193:57575)
0806 22:53:42.113781 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0806 22:53:42.113823 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 22:53:42.114356 (+ 533us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:53:42.114618 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:53:42.114622 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:53:42.115285 (+ 663us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:53:42.115466 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:53:42.115471 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:53:42.116210 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:53:42.116212 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:53:42.117070 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:53:42.117073 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:53:42.117497 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:53:42.117499 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:53:42.117626 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:42.118067 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:53:42.118074 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:53:42.119419 (+ 1345us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:53:42.121494 (+ 2075us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:53:42.121500 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:53:42.121502 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:53:42.121709 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:53:42.121924 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:53:42.121924 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:53:42.121924 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:53:42.122030 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:53:42.122319 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:53:42.122324 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:53:42.122426 (+ 102us) client_negotiation.cc:769] Sending connection context
0806 22:53:42.122567 (+ 141us) client_negotiation.cc:240] Negotiation successful
0806 22:53:42.122717 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":76,"threads_started":1}
I20260806 22:53:42.123584 16463 heartbeater.cc:347] Connected to a master server at 127.28.218.254:43035
I20260806 22:53:42.123687 16463 heartbeater.cc:464] Registering TS with master...
I20260806 22:53:42.123919 16463 heartbeater.cc:511] Master 127.28.218.254:43035 requested a full tablet report, sending...
I20260806 22:53:42.124413 16263 ts_manager.cc:194] Registered new tserver with Master: 2515e82235204a06a58e63d17fc1df96 (127.28.218.193:38227)
I20260806 22:53:42.125015 16263 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.218.193@KRBTEST.COM'} at 127.28.218.193:57575
WARNING: no policy specified for kudu/127.28.218.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.218.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.218.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.194.keytab.
Entry for principal kudu/127.28.218.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.194.keytab.
I20260806 22:53:42.154351 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056586788339-29547-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056586788339-29547-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056586788339-29547-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056586788339-29547-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.218.194:0
--local_ip_for_outbound_sockets=127.28.218.194
--webserver_interface=127.28.218.194
--webserver_port=0
--tserver_master_addrs=127.28.218.254:43035
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.194.keytab
--principal=kudu/127.28.218.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.218.212:38639
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:53:42.266608 16469 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:53:42.266821 16469 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:53:42.266845 16469 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:53:42.268074 16469 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:53:42.268118 16469 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:53:42.268144 16469 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.218.194
I20260806 22:53:42.269634 16469 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:38639
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056586788339-29547-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056586788339-29547-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.194.keytab
--principal=kudu/127.28.218.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.218.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056586788339-29547-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.218.194
--webserver_port=0
--tserver_master_addrs=127.28.218.254:43035
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16469
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.218.194
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056586788339-29547-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:53:42.269902 16469 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:53:42.270157 16469 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:53:42.275414 16478 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:53:42.275430 16476 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:53:42.275434 16475 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:53:42.275568 16469 server_base.cc:1034] running on GCE node
I20260806 22:53:42.275910 16469 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:53:42.276180 16469 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:53:42.277355 16469 hybrid_clock.cc:648] HybridClock initialized: now 1786056822277347 us; error 34 us; skew 500 ppm
Aug 06 22:53:42 dist-test-slave-1xrh krb5kdc[16215](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056822, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:53:42.280079 16469 init.cc:377] Logged in from keytab as kudu/127.28.218.194@KRBTEST.COM (short username kudu)
I20260806 22:53:42.280514 16469 webserver.cc:460] Webserver started at http://127.28.218.194:43187/ using document root <none> and password file <none>
I20260806 22:53:42.280747 16469 fs_manager.cc:359] Metadata directory not provided
I20260806 22:53:42.280794 16469 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:53:42.280957 16469 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:53:42.282006 16469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056586788339-29547-0/minicluster-data/ts-1/data/instance:
uuid: "a9e5454c627e40c893354fdd66eb2373"
format_stamp: "Formatted at 2026-08-06 22:53:42 on dist-test-slave-1xrh"
server_key: "d8fe1d07067dd7b4057ab0607ad2c25e"
server_key_iv: "fb02320f57ef2924b102e94577419f3e"
server_key_version: "encryptionkey@0"
I20260806 22:53:42.282373 16469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056586788339-29547-0/minicluster-data/ts-1/wal/instance:
uuid: "a9e5454c627e40c893354fdd66eb2373"
format_stamp: "Formatted at 2026-08-06 22:53:42 on dist-test-slave-1xrh"
server_key: "d8fe1d07067dd7b4057ab0607ad2c25e"
server_key_iv: "fb02320f57ef2924b102e94577419f3e"
server_key_version: "encryptionkey@0"
I20260806 22:53:42.283877 16469 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260806 22:53:42.284685 16485 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:53:42.284888 16469 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:53:42.285010 16469 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056586788339-29547-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056586788339-29547-0/minicluster-data/ts-1/wal
uuid: "a9e5454c627e40c893354fdd66eb2373"
format_stamp: "Formatted at 2026-08-06 22:53:42 on dist-test-slave-1xrh"
server_key: "d8fe1d07067dd7b4057ab0607ad2c25e"
server_key_iv: "fb02320f57ef2924b102e94577419f3e"
server_key_version: "encryptionkey@0"
I20260806 22:53:42.285092 16469 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056586788339-29547-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056586788339-29547-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056586788339-29547-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:53:42.296118 16469 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:53:42.298446 16469 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:53:42.298624 16469 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:53:42.298863 16469 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:53:42.299250 16469 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:53:42.299309 16469 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:53:42.299351 16469 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:53:42.299378 16469 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:53:42.305328 16469 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.194:40075
I20260806 22:53:42.305406 16602 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.194:40075 every 8 connection(s)
I20260806 22:53:42.305718 16469 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056586788339-29547-0/minicluster-data/ts-1/data/info.pb
I20260806 22:53:42.310285 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 16469
I20260806 22:53:42.310428 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056586788339-29547-0/minicluster-data/ts-1/wal/instance
I20260806 22:53:42.310678 29547 external_mini_cluster.cc:1428] Setting key f2d4372d2c57fd9e2f509a4a50f8e874
Aug 06 22:53:42 dist-test-slave-1xrh krb5kdc[16215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056822, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.194@KRBTEST.COM for kudu/127.28.218.254@KRBTEST.COM
I20260806 22:53:42.315789 16303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:42.306493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.194:36279 (local address 127.28.218.254:43035)
0806 22:53:42.306564 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0806 22:53:42.306566 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:53:42.306771 (+ 205us) server_negotiation.cc:407] Connection header received
0806 22:53:42.307369 (+ 598us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:53:42.307370 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:53:42.307380 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:53:42.307439 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:53:42.308337 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:42.308837 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:42.309923 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:42.310175 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:42.310257 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:42.312616 (+ 2359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:53:42.312617 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:53:42.312617 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:53:42.312626 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:53:42.314202 (+ 1576us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:53:42.314709 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:53:42.314710 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:53:42.314710 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:53:42.314760 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:53:42.315053 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:53:42.315054 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:53:42.315054 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:53:42.315263 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:53:42.315319 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 22:53:42.315662 (+ 343us) server_negotiation.cc:299] Negotiation successful
0806 22:53:42.315726 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 22:53:42.316092 16606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:42.306451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:43035 (local address 127.28.218.194:36279)
0806 22:53:42.306631 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0806 22:53:42.306672 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 22:53:42.307278 (+ 606us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:53:42.307491 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:53:42.307495 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:53:42.308051 (+ 556us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:53:42.308233 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:53:42.308237 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:53:42.308939 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:53:42.308939 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:53:42.309848 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:53:42.309850 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:53:42.310266 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:53:42.310267 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:53:42.310366 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:42.310949 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:53:42.310959 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:53:42.312521 (+ 1562us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:53:42.314294 (+ 1773us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:53:42.314300 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:53:42.314303 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:53:42.314630 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:53:42.314831 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:53:42.314832 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:53:42.314832 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:53:42.314969 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:53:42.315333 (+ 364us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:53:42.315340 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:53:42.315521 (+ 181us) client_negotiation.cc:769] Sending connection context
0806 22:53:42.315638 (+ 117us) client_negotiation.cc:240] Negotiation successful
0806 22:53:42.315812 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":90,"threads_started":1}
I20260806 22:53:42.316685 16603 heartbeater.cc:347] Connected to a master server at 127.28.218.254:43035
I20260806 22:53:42.316790 16603 heartbeater.cc:464] Registering TS with master...
I20260806 22:53:42.316972 16603 heartbeater.cc:511] Master 127.28.218.254:43035 requested a full tablet report, sending...
I20260806 22:53:42.317332 16263 ts_manager.cc:194] Registered new tserver with Master: a9e5454c627e40c893354fdd66eb2373 (127.28.218.194:40075)
I20260806 22:53:42.317761 16263 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.218.194@KRBTEST.COM'} at 127.28.218.194:36279
WARNING: no policy specified for kudu/127.28.218.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.218.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.218.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.195.keytab.
Entry for principal kudu/127.28.218.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.195.keytab.
I20260806 22:53:42.340315 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056586788339-29547-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056586788339-29547-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056586788339-29547-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056586788339-29547-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.218.195:0
--local_ip_for_outbound_sockets=127.28.218.195
--webserver_interface=127.28.218.195
--webserver_port=0
--tserver_master_addrs=127.28.218.254:43035
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.195.keytab
--principal=kudu/127.28.218.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.218.212:38639
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:53:42.425993 16609 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:53:42.426201 16609 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:53:42.426226 16609 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:53:42.427475 16609 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:53:42.427522 16609 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:53:42.427579 16609 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.218.195
I20260806 22:53:42.429097 16609 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:38639
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056586788339-29547-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056586788339-29547-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.195.keytab
--principal=kudu/127.28.218.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.218.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056586788339-29547-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.218.195
--webserver_port=0
--tserver_master_addrs=127.28.218.254:43035
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16609
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.218.195
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056586788339-29547-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:53:42.429323 16609 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:53:42.429548 16609 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:53:42.434769 16615 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:53:42.434768 16614 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:53:42.434791 16618 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:53:42.435104 16609 server_base.cc:1034] running on GCE node
I20260806 22:53:42.435297 16609 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:53:42.435488 16609 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:53:42.436650 16609 hybrid_clock.cc:648] HybridClock initialized: now 1786056822436639 us; error 30 us; skew 500 ppm
Aug 06 22:53:42 dist-test-slave-1xrh krb5kdc[16215](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056822, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:53:42.439047 16609 init.cc:377] Logged in from keytab as kudu/127.28.218.195@KRBTEST.COM (short username kudu)
I20260806 22:53:42.439492 16609 webserver.cc:460] Webserver started at http://127.28.218.195:45037/ using document root <none> and password file <none>
I20260806 22:53:42.439725 16609 fs_manager.cc:359] Metadata directory not provided
I20260806 22:53:42.439795 16609 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:53:42.439949 16609 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:53:42.440943 16609 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056586788339-29547-0/minicluster-data/ts-2/data/instance:
uuid: "53adbf4f47804c12ac573ef57e2e5013"
format_stamp: "Formatted at 2026-08-06 22:53:42 on dist-test-slave-1xrh"
server_key: "03c8478408f8f24f5febc5f90a48eb40"
server_key_iv: "c320ceab6c51cbaacc8c927eb6936094"
server_key_version: "encryptionkey@0"
I20260806 22:53:42.441308 16609 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056586788339-29547-0/minicluster-data/ts-2/wal/instance:
uuid: "53adbf4f47804c12ac573ef57e2e5013"
format_stamp: "Formatted at 2026-08-06 22:53:42 on dist-test-slave-1xrh"
server_key: "03c8478408f8f24f5febc5f90a48eb40"
server_key_iv: "c320ceab6c51cbaacc8c927eb6936094"
server_key_version: "encryptionkey@0"
I20260806 22:53:42.442695 16609 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 22:53:42.443475 16625 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:53:42.443653 16609 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 22:53:42.443753 16609 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056586788339-29547-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056586788339-29547-0/minicluster-data/ts-2/wal
uuid: "53adbf4f47804c12ac573ef57e2e5013"
format_stamp: "Formatted at 2026-08-06 22:53:42 on dist-test-slave-1xrh"
server_key: "03c8478408f8f24f5febc5f90a48eb40"
server_key_iv: "c320ceab6c51cbaacc8c927eb6936094"
server_key_version: "encryptionkey@0"
I20260806 22:53:42.443856 16609 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056586788339-29547-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056586788339-29547-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056586788339-29547-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:53:42.458971 16609 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:53:42.461027 16609 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:53:42.461197 16609 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:53:42.461419 16609 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:53:42.461719 16609 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:53:42.461773 16609 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:53:42.461814 16609 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:53:42.461866 16609 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:53:42.468196 16609 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.195:37439
I20260806 22:53:42.468242 16742 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.195:37439 every 8 connection(s)
I20260806 22:53:42.468542 16609 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056586788339-29547-0/minicluster-data/ts-2/data/info.pb
Aug 06 22:53:42 dist-test-slave-1xrh krb5kdc[16215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056822, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.195@KRBTEST.COM for kudu/127.28.218.254@KRBTEST.COM
I20260806 22:53:42.475499 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 16609
I20260806 22:53:42.475617 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056586788339-29547-0/minicluster-data/ts-2/wal/instance
I20260806 22:53:42.475850 29547 external_mini_cluster.cc:1428] Setting key 29e26dae22d2d86575c1efd32062c16a
I20260806 22:53:42.477675 16303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:42.469226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.195:46201 (local address 127.28.218.254:43035)
0806 22:53:42.469291 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0806 22:53:42.469293 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:53:42.469470 (+ 177us) server_negotiation.cc:407] Connection header received
0806 22:53:42.469934 (+ 464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:53:42.469936 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:53:42.469947 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:53:42.470006 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:53:42.470885 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:42.471556 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:42.472558 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:42.472743 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:42.472797 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:42.474756 (+ 1959us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:53:42.474757 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:53:42.474757 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:53:42.474763 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:53:42.476444 (+ 1681us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:53:42.476922 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:53:42.476923 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:53:42.476923 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:53:42.476976 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:53:42.477198 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:53:42.477199 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:53:42.477199 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:53:42.477297 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:53:42.477358 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 22:53:42.477555 (+ 197us) server_negotiation.cc:299] Negotiation successful
0806 22:53:42.477611 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 22:53:42.477917 16747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:42.469182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:43035 (local address 127.28.218.195:46201)
0806 22:53:42.469339 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0806 22:53:42.469380 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 22:53:42.469844 (+ 464us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:53:42.470057 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:53:42.470060 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:53:42.470609 (+ 549us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:53:42.470790 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:53:42.470795 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:53:42.471649 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:53:42.471650 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:53:42.472467 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:53:42.472470 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:53:42.472822 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:53:42.472823 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:53:42.472928 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:42.473358 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:53:42.473366 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:53:42.474674 (+ 1308us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:53:42.476547 (+ 1873us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:53:42.476554 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:53:42.476556 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:53:42.476856 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:53:42.477038 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:53:42.477039 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:53:42.477039 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:53:42.477143 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:53:42.477352 (+ 209us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:53:42.477356 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:53:42.477462 (+ 106us) client_negotiation.cc:769] Sending connection context
0806 22:53:42.477544 (+ 82us) client_negotiation.cc:240] Negotiation successful
0806 22:53:42.477657 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":87,"threads_started":1}
I20260806 22:53:42.478336 16744 heartbeater.cc:347] Connected to a master server at 127.28.218.254:43035
I20260806 22:53:42.478444 16744 heartbeater.cc:464] Registering TS with master...
I20260806 22:53:42.478626 16744 heartbeater.cc:511] Master 127.28.218.254:43035 requested a full tablet report, sending...
I20260806 22:53:42.479009 16263 ts_manager.cc:194] Registered new tserver with Master: 53adbf4f47804c12ac573ef57e2e5013 (127.28.218.195:37439)
I20260806 22:53:42.479404 16263 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.218.195@KRBTEST.COM'} at 127.28.218.195:46201
I20260806 22:53:42.487551 29547 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:53:42.494767 16303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:42.488937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45472 (local address 127.28.218.254:43035)
0806 22:53:42.488979 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 22:53:42.488980 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:53:42.489105 (+ 125us) server_negotiation.cc:407] Connection header received
0806 22:53:42.489145 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:53:42.489146 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:53:42.489155 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:53:42.489190 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:53:42.489725 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:42.490187 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:42.490861 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:42.491059 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:42.491158 (+ 99us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:42.492367 (+ 1209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:53:42.492368 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:53:42.492368 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:53:42.492374 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:53:42.493720 (+ 1346us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:53:42.494064 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:53:42.494065 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:53:42.494065 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:53:42.494114 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:53:42.494297 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:53:42.494298 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:53:42.494298 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:53:42.494417 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:53:42.494504 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 22:53:42.494595 (+ 91us) server_negotiation.cc:299] Negotiation successful
0806 22:53:42.494644 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 22:53:42.496696 16263 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45472:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:53:42.498451 16263 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:53:42.505203 16759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:42.500766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.194:40075 (local address 127.0.0.1:45778)
0806 22:53:42.500981 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0806 22:53:42.501019 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 22:53:42.501085 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:53:42.501344 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:53:42.501347 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:53:42.501349 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:53:42.501424 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:53:42.501426 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:53:42.502498 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:53:42.502500 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:53:42.504047 (+ 1547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:53:42.504051 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:53:42.504965 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:53:42.504966 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:53:42.505030 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:42.505034 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:53:42.505086 (+ 52us) client_negotiation.cc:240] Negotiation successful
0806 22:53:42.505128 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":143,"threads_started":1}
I20260806 22:53:42.505205 16760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:42.500919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37360 (local address 127.28.218.195:37439)
0806 22:53:42.501257 (+ 338us) server_negotiation.cc:206] Beginning negotiation
0806 22:53:42.501260 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:53:42.501268 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:53:42.501315 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:53:42.501317 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:53:42.501340 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:53:42.501423 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:53:42.501608 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:42.502639 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:42.504160 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:42.504756 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:42.504846 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:42.504901 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 22:53:42.505083 (+ 182us) server_negotiation.cc:299] Negotiation successful
0806 22:53:42.505121 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":303,"thread_start_us":80,"threads_started":1}
I20260806 22:53:42.505203 16757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:42.500766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.195:37439 (local address 127.0.0.1:37360)
0806 22:53:42.501076 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0806 22:53:42.501085 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:53:42.501128 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:53:42.501444 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:53:42.501445 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:53:42.501447 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:53:42.501514 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:53:42.501516 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:53:42.502744 (+ 1228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:53:42.502745 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:53:42.504065 (+ 1320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:53:42.504067 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:53:42.504851 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:53:42.504852 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:53:42.504950 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:42.504963 (+ 13us) client_negotiation.cc:769] Sending connection context
0806 22:53:42.505064 (+ 101us) client_negotiation.cc:240] Negotiation successful
0806 22:53:42.505122 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":298,"mutex_wait_us":4,"thread_start_us":46,"threads_started":1}
I20260806 22:53:42.505635 16762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:42.501003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46566 (local address 127.28.218.193:38227)
0806 22:53:42.501177 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0806 22:53:42.501181 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:53:42.501657 (+ 476us) server_negotiation.cc:407] Connection header received
0806 22:53:42.501702 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:53:42.501705 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:53:42.501720 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:53:42.501788 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:53:42.502011 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:42.502951 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:42.504503 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:42.505275 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:42.505380 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:42.505438 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 22:53:42.505489 (+ 51us) server_negotiation.cc:299] Negotiation successful
0806 22:53:42.505530 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":75,"threads_started":1}
I20260806 22:53:42.505544 16758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:42.500781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.193:38227 (local address 127.0.0.1:46566)
0806 22:53:42.501520 (+ 739us) negotiation.cc:107] Waiting for socket to connect
0806 22:53:42.501544 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 22:53:42.501595 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:53:42.501884 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:53:42.501885 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:53:42.501887 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:53:42.501921 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:53:42.501923 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:53:42.503031 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:53:42.503032 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:53:42.504387 (+ 1355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:53:42.504390 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:53:42.505362 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:53:42.505363 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:53:42.505426 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:42.505429 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:53:42.505476 (+ 47us) client_negotiation.cc:240] Negotiation successful
0806 22:53:42.505505 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":716,"mutex_wait_us":2,"thread_start_us":90,"threads_started":1}
I20260806 22:53:42.506165 16761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:42.500968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45778 (local address 127.28.218.194:40075)
0806 22:53:42.501166 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0806 22:53:42.501171 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:53:42.501182 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:53:42.501235 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:53:42.501236 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:53:42.501249 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:53:42.501320 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:53:42.501541 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:42.502398 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:42.504187 (+ 1789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:42.504808 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:42.504879 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:42.504928 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 22:53:42.505892 (+ 964us) server_negotiation.cc:299] Negotiation successful
0806 22:53:42.505928 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":84,"threads_started":1}
I20260806 22:53:42.506426 16676 tablet_service.cc:1467] Processing CreateTablet for tablet 2aa209b9ae07400288776b8b9d3957af (DEFAULT_TABLE table=test-table [id=f16f074077e642a3bc91bfa61aa76dfd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:53:42.506693 16395 tablet_service.cc:1467] Processing CreateTablet for tablet 2aa209b9ae07400288776b8b9d3957af (DEFAULT_TABLE table=test-table [id=f16f074077e642a3bc91bfa61aa76dfd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:53:42.506750 16535 tablet_service.cc:1467] Processing CreateTablet for tablet 2aa209b9ae07400288776b8b9d3957af (DEFAULT_TABLE table=test-table [id=f16f074077e642a3bc91bfa61aa76dfd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:53:42.506888 16676 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2aa209b9ae07400288776b8b9d3957af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:53:42.507038 16395 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2aa209b9ae07400288776b8b9d3957af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:53:42.507038 16535 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2aa209b9ae07400288776b8b9d3957af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:53:42.510637 16763 tablet_bootstrap.cc:492] T 2aa209b9ae07400288776b8b9d3957af P a9e5454c627e40c893354fdd66eb2373: Bootstrap starting.
I20260806 22:53:42.511670 16763 tablet_bootstrap.cc:654] T 2aa209b9ae07400288776b8b9d3957af P a9e5454c627e40c893354fdd66eb2373: Neither blocks nor log segments found. Creating new log.
I20260806 22:53:42.512238 16763 log.cc:824] T 2aa209b9ae07400288776b8b9d3957af P a9e5454c627e40c893354fdd66eb2373: Log is configured to *not* fsync() on all Append() calls
I20260806 22:53:42.513052 16766 tablet_bootstrap.cc:492] T 2aa209b9ae07400288776b8b9d3957af P 2515e82235204a06a58e63d17fc1df96: Bootstrap starting.
I20260806 22:53:42.513142 16763 tablet_bootstrap.cc:492] T 2aa209b9ae07400288776b8b9d3957af P a9e5454c627e40c893354fdd66eb2373: No bootstrap required, opened a new log
I20260806 22:53:42.513208 16763 ts_tablet_manager.cc:1397] T 2aa209b9ae07400288776b8b9d3957af P a9e5454c627e40c893354fdd66eb2373: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 22:53:42.514173 16766 tablet_bootstrap.cc:654] T 2aa209b9ae07400288776b8b9d3957af P 2515e82235204a06a58e63d17fc1df96: Neither blocks nor log segments found. Creating new log.
I20260806 22:53:42.514511 16764 tablet_bootstrap.cc:492] T 2aa209b9ae07400288776b8b9d3957af P 53adbf4f47804c12ac573ef57e2e5013: Bootstrap starting.
I20260806 22:53:42.514658 16766 log.cc:824] T 2aa209b9ae07400288776b8b9d3957af P 2515e82235204a06a58e63d17fc1df96: Log is configured to *not* fsync() on all Append() calls
I20260806 22:53:42.515056 16764 tablet_bootstrap.cc:654] T 2aa209b9ae07400288776b8b9d3957af P 53adbf4f47804c12ac573ef57e2e5013: Neither blocks nor log segments found. Creating new log.
I20260806 22:53:42.515589 16766 tablet_bootstrap.cc:492] T 2aa209b9ae07400288776b8b9d3957af P 2515e82235204a06a58e63d17fc1df96: No bootstrap required, opened a new log
I20260806 22:53:42.515611 16764 log.cc:824] T 2aa209b9ae07400288776b8b9d3957af P 53adbf4f47804c12ac573ef57e2e5013: Log is configured to *not* fsync() on all Append() calls
I20260806 22:53:42.515769 16766 ts_tablet_manager.cc:1397] T 2aa209b9ae07400288776b8b9d3957af P 2515e82235204a06a58e63d17fc1df96: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 22:53:42.515714 16763 raft_consensus.cc:348] T 2aa209b9ae07400288776b8b9d3957af P a9e5454c627e40c893354fdd66eb2373 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9e5454c627e40c893354fdd66eb2373" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 40075 } } peers { permanent_uuid: "53adbf4f47804c12ac573ef57e2e5013" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 37439 } } peers { permanent_uuid: "2515e82235204a06a58e63d17fc1df96" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 38227 } }
I20260806 22:53:42.515929 16763 raft_consensus.cc:374] T 2aa209b9ae07400288776b8b9d3957af P a9e5454c627e40c893354fdd66eb2373 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:53:42.515988 16763 raft_consensus.cc:729] T 2aa209b9ae07400288776b8b9d3957af P a9e5454c627e40c893354fdd66eb2373 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9e5454c627e40c893354fdd66eb2373, State: Initialized, Role: FOLLOWER
I20260806 22:53:42.516129 16763 consensus_queue.cc:260] T 2aa209b9ae07400288776b8b9d3957af P a9e5454c627e40c893354fdd66eb2373 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9e5454c627e40c893354fdd66eb2373" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 40075 } } peers { permanent_uuid: "53adbf4f47804c12ac573ef57e2e5013" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 37439 } } peers { permanent_uuid: "2515e82235204a06a58e63d17fc1df96" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 38227 } }
I20260806 22:53:42.516328 16764 tablet_bootstrap.cc:492] T 2aa209b9ae07400288776b8b9d3957af P 53adbf4f47804c12ac573ef57e2e5013: No bootstrap required, opened a new log
I20260806 22:53:42.516417 16764 ts_tablet_manager.cc:1397] T 2aa209b9ae07400288776b8b9d3957af P 53adbf4f47804c12ac573ef57e2e5013: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:53:42.516501 16763 ts_tablet_manager.cc:1428] T 2aa209b9ae07400288776b8b9d3957af P a9e5454c627e40c893354fdd66eb2373: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260806 22:53:42.516531 16603 heartbeater.cc:503] Master 127.28.218.254:43035 was elected leader, sending a full tablet report...
I20260806 22:53:42.518446 16764 raft_consensus.cc:348] T 2aa209b9ae07400288776b8b9d3957af P 53adbf4f47804c12ac573ef57e2e5013 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9e5454c627e40c893354fdd66eb2373" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 40075 } } peers { permanent_uuid: "53adbf4f47804c12ac573ef57e2e5013" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 37439 } } peers { permanent_uuid: "2515e82235204a06a58e63d17fc1df96" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 38227 } }
I20260806 22:53:42.518625 16764 raft_consensus.cc:374] T 2aa209b9ae07400288776b8b9d3957af P 53adbf4f47804c12ac573ef57e2e5013 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:53:42.518676 16764 raft_consensus.cc:729] T 2aa209b9ae07400288776b8b9d3957af P 53adbf4f47804c12ac573ef57e2e5013 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53adbf4f47804c12ac573ef57e2e5013, State: Initialized, Role: FOLLOWER
I20260806 22:53:42.518524 16766 raft_consensus.cc:348] T 2aa209b9ae07400288776b8b9d3957af P 2515e82235204a06a58e63d17fc1df96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9e5454c627e40c893354fdd66eb2373" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 40075 } } peers { permanent_uuid: "53adbf4f47804c12ac573ef57e2e5013" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 37439 } } peers { permanent_uuid: "2515e82235204a06a58e63d17fc1df96" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 38227 } }
I20260806 22:53:42.518798 16764 consensus_queue.cc:260] T 2aa209b9ae07400288776b8b9d3957af P 53adbf4f47804c12ac573ef57e2e5013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9e5454c627e40c893354fdd66eb2373" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 40075 } } peers { permanent_uuid: "53adbf4f47804c12ac573ef57e2e5013" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 37439 } } peers { permanent_uuid: "2515e82235204a06a58e63d17fc1df96" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 38227 } }
I20260806 22:53:42.518944 16766 raft_consensus.cc:374] T 2aa209b9ae07400288776b8b9d3957af P 2515e82235204a06a58e63d17fc1df96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:53:42.519057 16766 raft_consensus.cc:729] T 2aa209b9ae07400288776b8b9d3957af P 2515e82235204a06a58e63d17fc1df96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2515e82235204a06a58e63d17fc1df96, State: Initialized, Role: FOLLOWER
I20260806 22:53:42.519073 16744 heartbeater.cc:503] Master 127.28.218.254:43035 was elected leader, sending a full tablet report...
I20260806 22:53:42.519071 16764 ts_tablet_manager.cc:1428] T 2aa209b9ae07400288776b8b9d3957af P 53adbf4f47804c12ac573ef57e2e5013: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 22:53:42.519268 16766 consensus_queue.cc:260] T 2aa209b9ae07400288776b8b9d3957af P 2515e82235204a06a58e63d17fc1df96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9e5454c627e40c893354fdd66eb2373" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 40075 } } peers { permanent_uuid: "53adbf4f47804c12ac573ef57e2e5013" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 37439 } } peers { permanent_uuid: "2515e82235204a06a58e63d17fc1df96" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 38227 } }
I20260806 22:53:42.519580 16463 heartbeater.cc:503] Master 127.28.218.254:43035 was elected leader, sending a full tablet report...
I20260806 22:53:42.519579 16766 ts_tablet_manager.cc:1428] T 2aa209b9ae07400288776b8b9d3957af P 2515e82235204a06a58e63d17fc1df96: Time spent starting tablet: real 0.004s user 0.000s sys 0.003s
W20260806 22:53:42.556262 16605 tablet.cc:2369] T 2aa209b9ae07400288776b8b9d3957af P a9e5454c627e40c893354fdd66eb2373: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:53:42.579661 16771 raft_consensus.cc:482] T 2aa209b9ae07400288776b8b9d3957af P 2515e82235204a06a58e63d17fc1df96 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:53:42.579780 16771 raft_consensus.cc:504] T 2aa209b9ae07400288776b8b9d3957af P 2515e82235204a06a58e63d17fc1df96 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9e5454c627e40c893354fdd66eb2373" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 40075 } } peers { permanent_uuid: "53adbf4f47804c12ac573ef57e2e5013" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 37439 } } peers { permanent_uuid: "2515e82235204a06a58e63d17fc1df96" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 38227 } }
I20260806 22:53:42.580130 16771 leader_election.cc:288] T 2aa209b9ae07400288776b8b9d3957af P 2515e82235204a06a58e63d17fc1df96 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a9e5454c627e40c893354fdd66eb2373 (127.28.218.194:40075), 53adbf4f47804c12ac573ef57e2e5013 (127.28.218.195:37439)
I20260806 22:53:42.584110 16761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:42.580359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.193:40309 (local address 127.28.218.194:40075)
0806 22:53:42.580406 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 22:53:42.580408 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:53:42.580419 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:53:42.580448 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:53:42.580449 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:53:42.580459 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:53:42.580546 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:53:42.580767 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:42.581535 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:42.583092 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:42.583704 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:42.583812 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:42.583821 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 22:53:42.583996 (+ 175us) server_negotiation.cc:299] Negotiation successful
0806 22:53:42.584038 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 22:53:42.584110 16760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:42.580475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.193:52029 (local address 127.28.218.195:37439)
0806 22:53:42.580532 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0806 22:53:42.580534 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:53:42.580619 (+ 85us) server_negotiation.cc:407] Connection header received
0806 22:53:42.580642 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:53:42.580643 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:53:42.580650 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:53:42.580690 (+ 40us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:53:42.580862 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:42.581546 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:42.583090 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:42.583704 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:42.583812 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:42.583821 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 22:53:42.584005 (+ 184us) server_negotiation.cc:299] Negotiation successful
0806 22:53:42.584038 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260806 22:53:42.584125 16772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:42.580341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.195:37439 (local address 127.28.218.193:52029)
0806 22:53:42.580515 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0806 22:53:42.580526 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:53:42.580578 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:53:42.580718 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:53:42.580719 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:53:42.580721 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:53:42.580795 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:53:42.580796 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:53:42.581629 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:53:42.581630 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:53:42.582996 (+ 1366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:53:42.582999 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:53:42.583820 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:53:42.583821 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:53:42.583904 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:42.583911 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 22:53:42.583972 (+ 61us) client_negotiation.cc:240] Negotiation successful
0806 22:53:42.584017 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"mutex_wait_us":92,"thread_start_us":104,"threads_started":1}
I20260806 22:53:42.584136 16466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:42.580285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.194:40075 (local address 127.28.218.193:40309)
0806 22:53:42.580330 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0806 22:53:42.580343 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:53:42.580392 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:53:42.580574 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:53:42.580575 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:53:42.580577 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:53:42.580660 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:53:42.580662 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:53:42.581629 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:53:42.581630 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:53:42.582999 (+ 1369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:53:42.583001 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:53:42.583820 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:53:42.583821 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:53:42.583905 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:42.583911 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 22:53:42.583972 (+ 61us) client_negotiation.cc:240] Negotiation successful
0806 22:53:42.584017 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22,"mutex_wait_us":85}
I20260806 22:53:42.584758 16696 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2aa209b9ae07400288776b8b9d3957af" candidate_uuid: "2515e82235204a06a58e63d17fc1df96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53adbf4f47804c12ac573ef57e2e5013" is_pre_election: true
I20260806 22:53:42.584792 16557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2aa209b9ae07400288776b8b9d3957af" candidate_uuid: "2515e82235204a06a58e63d17fc1df96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9e5454c627e40c893354fdd66eb2373" is_pre_election: true
I20260806 22:53:42.584925 16696 raft_consensus.cc:2454] T 2aa209b9ae07400288776b8b9d3957af P 53adbf4f47804c12ac573ef57e2e5013 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2515e82235204a06a58e63d17fc1df96 in term 0.
I20260806 22:53:42.584925 16557 raft_consensus.cc:2454] T 2aa209b9ae07400288776b8b9d3957af P a9e5454c627e40c893354fdd66eb2373 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2515e82235204a06a58e63d17fc1df96 in term 0.
I20260806 22:53:42.585192 16347 leader_election.cc:302] T 2aa209b9ae07400288776b8b9d3957af P 2515e82235204a06a58e63d17fc1df96 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2515e82235204a06a58e63d17fc1df96, 53adbf4f47804c12ac573ef57e2e5013; no voters:
I20260806 22:53:42.585353 16771 raft_consensus.cc:2781] T 2aa209b9ae07400288776b8b9d3957af P 2515e82235204a06a58e63d17fc1df96 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:53:42.585428 16771 raft_consensus.cc:482] T 2aa209b9ae07400288776b8b9d3957af P 2515e82235204a06a58e63d17fc1df96 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:53:42.585446 16771 raft_consensus.cc:3037] T 2aa209b9ae07400288776b8b9d3957af P 2515e82235204a06a58e63d17fc1df96 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:53:42.586189 16771 raft_consensus.cc:504] T 2aa209b9ae07400288776b8b9d3957af P 2515e82235204a06a58e63d17fc1df96 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9e5454c627e40c893354fdd66eb2373" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 40075 } } peers { permanent_uuid: "53adbf4f47804c12ac573ef57e2e5013" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 37439 } } peers { permanent_uuid: "2515e82235204a06a58e63d17fc1df96" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 38227 } }
I20260806 22:53:42.586330 16771 leader_election.cc:288] T 2aa209b9ae07400288776b8b9d3957af P 2515e82235204a06a58e63d17fc1df96 [CANDIDATE]: Term 1 election: Requested vote from peers a9e5454c627e40c893354fdd66eb2373 (127.28.218.194:40075), 53adbf4f47804c12ac573ef57e2e5013 (127.28.218.195:37439)
I20260806 22:53:42.586508 16557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2aa209b9ae07400288776b8b9d3957af" candidate_uuid: "2515e82235204a06a58e63d17fc1df96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9e5454c627e40c893354fdd66eb2373"
I20260806 22:53:42.586498 16696 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2aa209b9ae07400288776b8b9d3957af" candidate_uuid: "2515e82235204a06a58e63d17fc1df96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53adbf4f47804c12ac573ef57e2e5013"
I20260806 22:53:42.586604 16696 raft_consensus.cc:3037] T 2aa209b9ae07400288776b8b9d3957af P 53adbf4f47804c12ac573ef57e2e5013 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:53:42.586606 16557 raft_consensus.cc:3037] T 2aa209b9ae07400288776b8b9d3957af P a9e5454c627e40c893354fdd66eb2373 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:53:42.587613 16696 raft_consensus.cc:2454] T 2aa209b9ae07400288776b8b9d3957af P 53adbf4f47804c12ac573ef57e2e5013 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2515e82235204a06a58e63d17fc1df96 in term 1.
I20260806 22:53:42.587613 16557 raft_consensus.cc:2454] T 2aa209b9ae07400288776b8b9d3957af P a9e5454c627e40c893354fdd66eb2373 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2515e82235204a06a58e63d17fc1df96 in term 1.
I20260806 22:53:42.587783 16350 leader_election.cc:302] T 2aa209b9ae07400288776b8b9d3957af P 2515e82235204a06a58e63d17fc1df96 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2515e82235204a06a58e63d17fc1df96, a9e5454c627e40c893354fdd66eb2373; no voters:
I20260806 22:53:42.587929 16771 raft_consensus.cc:2781] T 2aa209b9ae07400288776b8b9d3957af P 2515e82235204a06a58e63d17fc1df96 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:53:42.588155 16771 raft_consensus.cc:686] T 2aa209b9ae07400288776b8b9d3957af P 2515e82235204a06a58e63d17fc1df96 [term 1 LEADER]: Becoming Leader. State: Replica: 2515e82235204a06a58e63d17fc1df96, State: Running, Role: LEADER
I20260806 22:53:42.588243 16771 consensus_queue.cc:237] T 2aa209b9ae07400288776b8b9d3957af P 2515e82235204a06a58e63d17fc1df96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9e5454c627e40c893354fdd66eb2373" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 40075 } } peers { permanent_uuid: "53adbf4f47804c12ac573ef57e2e5013" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 37439 } } peers { permanent_uuid: "2515e82235204a06a58e63d17fc1df96" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 38227 } }
I20260806 22:53:42.589174 16261 catalog_manager.cc:5495] T 2aa209b9ae07400288776b8b9d3957af P 2515e82235204a06a58e63d17fc1df96 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2515e82235204a06a58e63d17fc1df96 (127.28.218.193). New cstate: current_term: 1 leader_uuid: "2515e82235204a06a58e63d17fc1df96" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9e5454c627e40c893354fdd66eb2373" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 40075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53adbf4f47804c12ac573ef57e2e5013" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 37439 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2515e82235204a06a58e63d17fc1df96" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 38227 } health_report { overall_health: HEALTHY } } }
I20260806 22:53:42.598300 16762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:42.596306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46570 (local address 127.28.218.193:38227)
0806 22:53:42.596344 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0806 22:53:42.596346 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:53:42.596402 (+ 56us) server_negotiation.cc:407] Connection header received
0806 22:53:42.596446 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:53:42.596446 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:53:42.596460 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:53:42.596520 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:53:42.596625 (+ 105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:42.597095 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:42.597751 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:42.597917 (+ 166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:42.597962 (+ 45us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:42.598090 (+ 128us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:53:42.598165 (+ 75us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:53:42.598229 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0806 22:53:42.598248 (+ 19us) server_negotiation.cc:299] Negotiation successful
0806 22:53:42.598256 (+ 8us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 22:53:42.599893 16557 raft_consensus.cc:1261] T 2aa209b9ae07400288776b8b9d3957af P a9e5454c627e40c893354fdd66eb2373 [term 1 FOLLOWER]: Refusing update from remote peer 2515e82235204a06a58e63d17fc1df96: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:53:42.599893 16696 raft_consensus.cc:1261] T 2aa209b9ae07400288776b8b9d3957af P 53adbf4f47804c12ac573ef57e2e5013 [term 1 FOLLOWER]: Refusing update from remote peer 2515e82235204a06a58e63d17fc1df96: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:53:42.600204 16771 consensus_queue.cc:1035] T 2aa209b9ae07400288776b8b9d3957af P 2515e82235204a06a58e63d17fc1df96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9e5454c627e40c893354fdd66eb2373" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 40075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:53:42.600361 16774 consensus_queue.cc:1035] T 2aa209b9ae07400288776b8b9d3957af P 2515e82235204a06a58e63d17fc1df96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53adbf4f47804c12ac573ef57e2e5013" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 37439 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:53:42.603024 16780 mvcc.cc:205] Tried to move back new op lower bound from 7315688745366790144 to 7315688745322053632. Current Snapshot: MvccSnapshot[applied={T|T < 7315688745366790144}]
I20260806 22:53:42.605101 16261 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45472:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:53:42.605185 16261 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45472:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:53:42.605929 16261 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5757ffaa36f54563b9caf32ba61be811: Sending DeleteTablet for 3 replicas of tablet 2aa209b9ae07400288776b8b9d3957af
I20260806 22:53:42.606280 16395 tablet_service.cc:1514] Processing DeleteTablet for tablet 2aa209b9ae07400288776b8b9d3957af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:53:42 UTC) from {username='kudu', principal='kudu/127.28.218.254@KRBTEST.COM'} at 127.0.0.1:46566
I20260806 22:53:42.606285 16535 tablet_service.cc:1514] Processing DeleteTablet for tablet 2aa209b9ae07400288776b8b9d3957af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:53:42 UTC) from {username='kudu', principal='kudu/127.28.218.254@KRBTEST.COM'} at 127.0.0.1:45778
I20260806 22:53:42.606302 16676 tablet_service.cc:1514] Processing DeleteTablet for tablet 2aa209b9ae07400288776b8b9d3957af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:53:42 UTC) from {username='kudu', principal='kudu/127.28.218.254@KRBTEST.COM'} at 127.0.0.1:37360
I20260806 22:53:42.606551 16789 tablet_replica.cc:330] T 2aa209b9ae07400288776b8b9d3957af P 53adbf4f47804c12ac573ef57e2e5013: stopping tablet replica
I20260806 22:53:42.606642 16789 raft_consensus.cc:2229] T 2aa209b9ae07400288776b8b9d3957af P 53adbf4f47804c12ac573ef57e2e5013 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:53:42.606734 16789 raft_consensus.cc:2258] T 2aa209b9ae07400288776b8b9d3957af P 53adbf4f47804c12ac573ef57e2e5013 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:53:42.606813 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 16327
I20260806 22:53:42.606864 16790 tablet_replica.cc:330] T 2aa209b9ae07400288776b8b9d3957af P a9e5454c627e40c893354fdd66eb2373: stopping tablet replica
I20260806 22:53:42.608740 16790 raft_consensus.cc:2229] T 2aa209b9ae07400288776b8b9d3957af P a9e5454c627e40c893354fdd66eb2373 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:53:42.608868 16790 raft_consensus.cc:2258] T 2aa209b9ae07400288776b8b9d3957af P a9e5454c627e40c893354fdd66eb2373 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:53:42.609359 16789 ts_tablet_manager.cc:1905] T 2aa209b9ae07400288776b8b9d3957af P 53adbf4f47804c12ac573ef57e2e5013: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:53:42.609488 16790 ts_tablet_manager.cc:1905] T 2aa209b9ae07400288776b8b9d3957af P a9e5454c627e40c893354fdd66eb2373: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:53:42.611166 16790 ts_tablet_manager.cc:1918] T 2aa209b9ae07400288776b8b9d3957af P a9e5454c627e40c893354fdd66eb2373: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:53:42.611249 16790 log.cc:1196] T 2aa209b9ae07400288776b8b9d3957af P a9e5454c627e40c893354fdd66eb2373: Deleting WAL directory at /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056586788339-29547-0/minicluster-data/ts-1/wal/wals/2aa209b9ae07400288776b8b9d3957af
I20260806 22:53:42.611357 16789 ts_tablet_manager.cc:1918] T 2aa209b9ae07400288776b8b9d3957af P 53adbf4f47804c12ac573ef57e2e5013: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:53:42.611434 16789 log.cc:1196] T 2aa209b9ae07400288776b8b9d3957af P 53adbf4f47804c12ac573ef57e2e5013: Deleting WAL directory at /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056586788339-29547-0/minicluster-data/ts-2/wal/wals/2aa209b9ae07400288776b8b9d3957af
I20260806 22:53:42.611598 16790 ts_tablet_manager.cc:1939] T 2aa209b9ae07400288776b8b9d3957af P a9e5454c627e40c893354fdd66eb2373: Deleting consensus metadata
I20260806 22:53:42.611711 16789 ts_tablet_manager.cc:1939] T 2aa209b9ae07400288776b8b9d3957af P 53adbf4f47804c12ac573ef57e2e5013: Deleting consensus metadata
I20260806 22:53:42.612049 16248 catalog_manager.cc:4841] TS a9e5454c627e40c893354fdd66eb2373 (127.28.218.194:40075): tablet 2aa209b9ae07400288776b8b9d3957af (table test-table [id=f16f074077e642a3bc91bfa61aa76dfd]) successfully deleted
I20260806 22:53:42.612244 16245 catalog_manager.cc:4841] TS 53adbf4f47804c12ac573ef57e2e5013 (127.28.218.195:37439): tablet 2aa209b9ae07400288776b8b9d3957af (table test-table [id=f16f074077e642a3bc91bfa61aa76dfd]) successfully deleted
W20260806 22:53:42.614146 16247 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.28.218.193:38227 (error 108)
I20260806 22:53:42.614622 16758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:42.614536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.193:38227 (local address 127.0.0.1:46578)
0806 22:53:42.614568 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0806 22:53:42.614598 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.193:38227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
W20260806 22:53:42.614735 16247 catalog_manager.cc:4584] TS 2515e82235204a06a58e63d17fc1df96 (127.28.218.193:38227): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2aa209b9ae07400288776b8b9d3957af: Network error: Client connection negotiation failed: client connection to 127.28.218.193:38227: connect: Connection refused (error 111)
I20260806 22:53:42.614758 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 16469
I20260806 22:53:42.623063 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 16609
I20260806 22:53:42.628770 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 16228
2026-08-06T22:53:42Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (1997 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-06T22:53:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:53:42Z Disabled control of system clock
I20260806 22:53:42.658820 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056586788339-29547-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056586788339-29547-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056586788339-29547-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.218.254:41181
--webserver_interface=127.28.218.254
--webserver_port=0
--builtin_ntp_servers=127.28.218.212:44869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.218.254:41181
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 22:53:42.744664 16796 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:53:42.744915 16796 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:53:42.744937 16796 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:53:42.746079 16796 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:53:42.746117 16796 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:53:42.746129 16796 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:53:42.746142 16796 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:53:42.746155 16796 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:53:42.747453 16796 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:44869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056586788339-29547-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056586788339-29547-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.218.254:41181
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.218.254:41181
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.218.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16796
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056586788339-29547-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:53:42.747658 16796 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:53:42.747905 16796 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:53:42.753314 16804 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:53:42.753333 16801 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:53:42.753440 16796 server_base.cc:1034] running on GCE node
W20260806 22:53:42.753350 16802 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:53:42.754055 16796 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:53:42.754348 16796 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:53:42.755520 16796 hybrid_clock.cc:648] HybridClock initialized: now 1786056822755515 us; error 34 us; skew 500 ppm
I20260806 22:53:42.756865 16796 webserver.cc:460] Webserver started at http://127.28.218.254:40751/ using document root <none> and password file <none>
I20260806 22:53:42.757118 16796 fs_manager.cc:359] Metadata directory not provided
I20260806 22:53:42.757191 16796 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:53:42.757344 16796 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:53:42.758404 16796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056586788339-29547-0/minicluster-data/master-0/data/instance:
uuid: "5dddfc4701414b10acab77047a97cf61"
format_stamp: "Formatted at 2026-08-06 22:53:42 on dist-test-slave-1xrh"
server_key: "e7881c2aac2620a3dcf4ff91eab00d19"
server_key_iv: "4e5a0ffb9d542f9a735b41d317d817ff"
server_key_version: "encryptionkey@0"
I20260806 22:53:42.758762 16796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056586788339-29547-0/minicluster-data/master-0/wal/instance:
uuid: "5dddfc4701414b10acab77047a97cf61"
format_stamp: "Formatted at 2026-08-06 22:53:42 on dist-test-slave-1xrh"
server_key: "e7881c2aac2620a3dcf4ff91eab00d19"
server_key_iv: "4e5a0ffb9d542f9a735b41d317d817ff"
server_key_version: "encryptionkey@0"
I20260806 22:53:42.760202 16796 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:53:42.760977 16810 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:53:42.761183 16796 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:53:42.761284 16796 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056586788339-29547-0/minicluster-data/master-0/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056586788339-29547-0/minicluster-data/master-0/wal
uuid: "5dddfc4701414b10acab77047a97cf61"
format_stamp: "Formatted at 2026-08-06 22:53:42 on dist-test-slave-1xrh"
server_key: "e7881c2aac2620a3dcf4ff91eab00d19"
server_key_iv: "4e5a0ffb9d542f9a735b41d317d817ff"
server_key_version: "encryptionkey@0"
I20260806 22:53:42.761364 16796 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056586788339-29547-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056586788339-29547-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056586788339-29547-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:53:42.774956 16796 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:53:42.775372 16796 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:53:42.775537 16796 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:53:42.779569 16796 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.254:41181
I20260806 22:53:42.779615 16864 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.254:41181 every 8 connection(s)
I20260806 22:53:42.779954 16796 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
I20260806 22:53:42.780980 16865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:53:42.783351 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 16796
I20260806 22:53:42.783481 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056586788339-29547-0/minicluster-data/master-0/wal/instance
I20260806 22:53:42.783483 16865 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5dddfc4701414b10acab77047a97cf61: Bootstrap starting.
I20260806 22:53:42.783690 29547 external_mini_cluster.cc:1428] Setting key cda23600860c0a89f6ded5bbc09a2733
I20260806 22:53:42.784284 16865 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5dddfc4701414b10acab77047a97cf61: Neither blocks nor log segments found. Creating new log.
I20260806 22:53:42.784634 16865 log.cc:824] T 00000000000000000000000000000000 P 5dddfc4701414b10acab77047a97cf61: Log is configured to *not* fsync() on all Append() calls
I20260806 22:53:42.785427 16865 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5dddfc4701414b10acab77047a97cf61: No bootstrap required, opened a new log
I20260806 22:53:42.787559 16865 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5dddfc4701414b10acab77047a97cf61 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5dddfc4701414b10acab77047a97cf61" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 41181 } }
I20260806 22:53:42.787766 16865 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5dddfc4701414b10acab77047a97cf61 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:53:42.787822 16865 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5dddfc4701414b10acab77047a97cf61 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5dddfc4701414b10acab77047a97cf61, State: Initialized, Role: FOLLOWER
I20260806 22:53:42.787966 16865 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5dddfc4701414b10acab77047a97cf61 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5dddfc4701414b10acab77047a97cf61" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 41181 } }
I20260806 22:53:42.788053 16865 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5dddfc4701414b10acab77047a97cf61 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:53:42.788098 16865 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5dddfc4701414b10acab77047a97cf61 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:53:42.788147 16865 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5dddfc4701414b10acab77047a97cf61 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:53:42.788899 16865 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5dddfc4701414b10acab77047a97cf61 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5dddfc4701414b10acab77047a97cf61" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 41181 } }
I20260806 22:53:42.789048 16865 leader_election.cc:302] T 00000000000000000000000000000000 P 5dddfc4701414b10acab77047a97cf61 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5dddfc4701414b10acab77047a97cf61; no voters:
I20260806 22:53:42.789222 16865 leader_election.cc:288] T 00000000000000000000000000000000 P 5dddfc4701414b10acab77047a97cf61 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:53:42.789264 16870 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5dddfc4701414b10acab77047a97cf61 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:53:42.789417 16870 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5dddfc4701414b10acab77047a97cf61 [term 1 LEADER]: Becoming Leader. State: Replica: 5dddfc4701414b10acab77047a97cf61, State: Running, Role: LEADER
I20260806 22:53:42.789484 16865 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5dddfc4701414b10acab77047a97cf61 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:53:42.789608 16870 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5dddfc4701414b10acab77047a97cf61 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5dddfc4701414b10acab77047a97cf61" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 41181 } }
I20260806 22:53:42.789971 16870 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5dddfc4701414b10acab77047a97cf61 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5dddfc4701414b10acab77047a97cf61. Latest consensus state: current_term: 1 leader_uuid: "5dddfc4701414b10acab77047a97cf61" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5dddfc4701414b10acab77047a97cf61" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 41181 } } }
I20260806 22:53:42.790040 16868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:42.784257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40606 (local address 127.28.218.254:41181)
0806 22:53:42.784523 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0806 22:53:42.784530 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:53:42.784556 (+ 26us) server_negotiation.cc:407] Connection header received
0806 22:53:42.785137 (+ 581us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:53:42.785147 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:53:42.785218 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:53:42.785333 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:53:42.785614 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:42.786502 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:42.787683 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:42.788277 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:42.788369 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:42.789073 (+ 704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:53:42.789077 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:53:42.789078 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:53:42.789297 (+ 219us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:53:42.789343 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:53:42.789360 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:53:42.789701 (+ 341us) server_negotiation.cc:1035] Waiting for connection context
0806 22:53:42.789732 (+ 31us) server_negotiation.cc:299] Negotiation successful
0806 22:53:42.789810 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":131,"threads_started":1}
I20260806 22:53:42.789983 16871 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5dddfc4701414b10acab77047a97cf61 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5dddfc4701414b10acab77047a97cf61" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5dddfc4701414b10acab77047a97cf61" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 41181 } } }
I20260806 22:53:42.790143 16870 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5dddfc4701414b10acab77047a97cf61 [sys.catalog]: This master's current role is: LEADER
I20260806 22:53:42.790172 16871 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5dddfc4701414b10acab77047a97cf61 [sys.catalog]: This master's current role is: LEADER
I20260806 22:53:42.790861 16877 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:53:42.791431 16877 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:53:42.792907 16877 catalog_manager.cc:1347] Generated new cluster ID: 3a4ccbbbb4684f58af21103526364239
I20260806 22:53:42.792951 16877 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:53:42.804106 16877 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:53:42.804633 16877 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:53:42.808251 16877 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5dddfc4701414b10acab77047a97cf61: Generated new TSK 0
I20260806 22:53:42.808434 16877 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:53:42.813942 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056586788339-29547-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056586788339-29547-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056586788339-29547-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056586788339-29547-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.218.193:0
--local_ip_for_outbound_sockets=127.28.218.193
--webserver_interface=127.28.218.193
--webserver_port=0
--tserver_master_addrs=127.28.218.254:41181
--builtin_ntp_servers=127.28.218.212:44869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 22:53:42.896157 16889 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:53:42.896385 16889 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:53:42.896421 16889 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:53:42.897535 16889 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:53:42.897596 16889 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:53:42.897637 16889 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.218.193
I20260806 22:53:42.898962 16889 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:44869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056586788339-29547-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056586788339-29547-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.218.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056586788339-29547-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.218.193
--webserver_port=0
--tserver_master_addrs=127.28.218.254:41181
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16889
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.218.193
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056586788339-29547-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:53:42.899276 16889 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:53:42.899578 16889 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:53:42.905284 16894 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:53:42.905346 16895 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:53:42.905335 16897 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:53:42.905608 16889 server_base.cc:1034] running on GCE node
I20260806 22:53:42.905802 16889 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:53:42.906090 16889 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:53:42.907181 16889 hybrid_clock.cc:648] HybridClock initialized: now 1786056822907145 us; error 52 us; skew 500 ppm
I20260806 22:53:42.908507 16889 webserver.cc:460] Webserver started at http://127.28.218.193:45803/ using document root <none> and password file <none>
I20260806 22:53:42.908759 16889 fs_manager.cc:359] Metadata directory not provided
I20260806 22:53:42.908841 16889 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:53:42.908974 16889 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:53:42.910032 16889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056586788339-29547-0/minicluster-data/ts-0/data/instance:
uuid: "4f3687580aff44ca991bebd065e20d15"
format_stamp: "Formatted at 2026-08-06 22:53:42 on dist-test-slave-1xrh"
server_key: "731c3b05c0ff996084e88622c0e429f9"
server_key_iv: "edae7e12f56a478e17db456f55e14746"
server_key_version: "encryptionkey@0"
I20260806 22:53:42.910398 16889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056586788339-29547-0/minicluster-data/ts-0/wal/instance:
uuid: "4f3687580aff44ca991bebd065e20d15"
format_stamp: "Formatted at 2026-08-06 22:53:42 on dist-test-slave-1xrh"
server_key: "731c3b05c0ff996084e88622c0e429f9"
server_key_iv: "edae7e12f56a478e17db456f55e14746"
server_key_version: "encryptionkey@0"
I20260806 22:53:42.911856 16889 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:53:42.912634 16903 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:53:42.912788 16889 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:53:42.912873 16889 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056586788339-29547-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056586788339-29547-0/minicluster-data/ts-0/wal
uuid: "4f3687580aff44ca991bebd065e20d15"
format_stamp: "Formatted at 2026-08-06 22:53:42 on dist-test-slave-1xrh"
server_key: "731c3b05c0ff996084e88622c0e429f9"
server_key_iv: "edae7e12f56a478e17db456f55e14746"
server_key_version: "encryptionkey@0"
I20260806 22:53:42.912945 16889 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056586788339-29547-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056586788339-29547-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056586788339-29547-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:53:42.929505 16889 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:53:42.929809 16889 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:53:42.929961 16889 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:53:42.930197 16889 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:53:42.930526 16889 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:53:42.930564 16889 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:53:42.930585 16889 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:53:42.930627 16889 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:53:42.936815 16889 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.193:42603
I20260806 22:53:42.936856 17025 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.193:42603 every 8 connection(s)
I20260806 22:53:42.937194 16889 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056586788339-29547-0/minicluster-data/ts-0/data/info.pb
I20260806 22:53:42.939711 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 16889
I20260806 22:53:42.939849 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056586788339-29547-0/minicluster-data/ts-0/wal/instance
I20260806 22:53:42.940055 29547 external_mini_cluster.cc:1428] Setting key 5936112fead5b34aaec2ac08eace03d3
I20260806 22:53:42.941124 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056586788339-29547-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056586788339-29547-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056586788339-29547-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056586788339-29547-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.218.194:0
--local_ip_for_outbound_sockets=127.28.218.194
--webserver_interface=127.28.218.194
--webserver_port=0
--tserver_master_addrs=127.28.218.254:41181
--builtin_ntp_servers=127.28.218.212:44869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 22:53:42.943490 16868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:42.938053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.193:37237 (local address 127.28.218.254:41181)
0806 22:53:42.938123 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0806 22:53:42.938125 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:53:42.938364 (+ 239us) server_negotiation.cc:407] Connection header received
0806 22:53:42.938924 (+ 560us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:53:42.938925 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:53:42.938941 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:53:42.939009 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:53:42.939742 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:42.940463 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:42.941940 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:42.942131 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:42.942182 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:42.943188 (+ 1006us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:53:42.943189 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:53:42.943190 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:53:42.943195 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:53:42.943212 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:53:42.943216 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:53:42.943273 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 22:53:42.943364 (+ 91us) server_negotiation.cc:299] Negotiation successful
0806 22:53:42.943389 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 22:53:42.943636 17028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:42.938039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:41181 (local address 127.28.218.193:37237)
0806 22:53:42.938239 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0806 22:53:42.938273 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0806 22:53:42.938793 (+ 520us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:53:42.939079 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:53:42.939083 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:53:42.939091 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:53:42.939641 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:53:42.939645 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:53:42.940553 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:53:42.940554 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:53:42.941839 (+ 1285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:53:42.941842 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:53:42.942219 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:53:42.942220 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:53:42.942385 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:42.943015 (+ 630us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:53:42.943016 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:53:42.943045 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:53:42.943067 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:53:42.943281 (+ 214us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:53:42.943286 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:53:42.943289 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:53:42.943342 (+ 53us) client_negotiation.cc:240] Negotiation successful
0806 22:53:42.943421 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":96,"threads_started":1}
I20260806 22:53:42.944198 17026 heartbeater.cc:347] Connected to a master server at 127.28.218.254:41181
I20260806 22:53:42.944289 17026 heartbeater.cc:464] Registering TS with master...
I20260806 22:53:42.944487 17026 heartbeater.cc:511] Master 127.28.218.254:41181 requested a full tablet report, sending...
I20260806 22:53:42.945006 16827 ts_manager.cc:194] Registered new tserver with Master: 4f3687580aff44ca991bebd065e20d15 (127.28.218.193:42603)
I20260806 22:53:42.945667 16827 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.28.218.193:37237
W20260806 22:53:43.023301 17029 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:53:43.023538 17029 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:53:43.023574 17029 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:53:43.024722 17029 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:53:43.024791 17029 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:53:43.024830 17029 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.218.194
I20260806 22:53:43.026201 17029 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:44869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056586788339-29547-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056586788339-29547-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.218.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056586788339-29547-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.218.194
--webserver_port=0
--tserver_master_addrs=127.28.218.254:41181
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17029
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.218.194
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056586788339-29547-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:53:43.026508 17029 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:53:43.026811 17029 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 22:53:43.031908 17029 server_base.cc:1034] running on GCE node
W20260806 22:53:43.031963 17034 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:53:43.031940 17038 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:53:43.031931 17036 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:53:43.032457 17029 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:53:43.032709 17029 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:53:43.033877 17029 hybrid_clock.cc:648] HybridClock initialized: now 1786056823033867 us; error 28 us; skew 500 ppm
I20260806 22:53:43.035230 17029 webserver.cc:460] Webserver started at http://127.28.218.194:45565/ using document root <none> and password file <none>
I20260806 22:53:43.035496 17029 fs_manager.cc:359] Metadata directory not provided
I20260806 22:53:43.035585 17029 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:53:43.035723 17029 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:53:43.036748 17029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056586788339-29547-0/minicluster-data/ts-1/data/instance:
uuid: "1f33d2fff7834cc3a621806f49ef1296"
format_stamp: "Formatted at 2026-08-06 22:53:43 on dist-test-slave-1xrh"
server_key: "8651d21938f9875654bf5417a8bfbb0e"
server_key_iv: "ac4ff92645719ed488a33d3ca32b1c3d"
server_key_version: "encryptionkey@0"
I20260806 22:53:43.037117 17029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056586788339-29547-0/minicluster-data/ts-1/wal/instance:
uuid: "1f33d2fff7834cc3a621806f49ef1296"
format_stamp: "Formatted at 2026-08-06 22:53:43 on dist-test-slave-1xrh"
server_key: "8651d21938f9875654bf5417a8bfbb0e"
server_key_iv: "ac4ff92645719ed488a33d3ca32b1c3d"
server_key_version: "encryptionkey@0"
I20260806 22:53:43.038391 17029 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:53:43.039052 17044 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:53:43.039239 17029 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:53:43.039336 17029 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056586788339-29547-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056586788339-29547-0/minicluster-data/ts-1/wal
uuid: "1f33d2fff7834cc3a621806f49ef1296"
format_stamp: "Formatted at 2026-08-06 22:53:43 on dist-test-slave-1xrh"
server_key: "8651d21938f9875654bf5417a8bfbb0e"
server_key_iv: "ac4ff92645719ed488a33d3ca32b1c3d"
server_key_version: "encryptionkey@0"
I20260806 22:53:43.039417 17029 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056586788339-29547-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056586788339-29547-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056586788339-29547-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:53:43.055065 17029 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:53:43.055349 17029 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:53:43.055496 17029 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:53:43.055704 17029 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:53:43.056010 17029 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:53:43.056066 17029 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:53:43.056105 17029 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:53:43.056130 17029 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:53:43.062183 17029 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.194:36979
I20260806 22:53:43.062227 17165 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.194:36979 every 8 connection(s)
I20260806 22:53:43.062539 17029 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056586788339-29547-0/minicluster-data/ts-1/data/info.pb
I20260806 22:53:43.067373 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 17029
I20260806 22:53:43.067502 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056586788339-29547-0/minicluster-data/ts-1/wal/instance
I20260806 22:53:43.067566 16868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:43.063242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.194:36429 (local address 127.28.218.254:41181)
0806 22:53:43.063295 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0806 22:53:43.063297 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:53:43.063515 (+ 218us) server_negotiation.cc:407] Connection header received
0806 22:53:43.063987 (+ 472us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:53:43.063988 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:53:43.063997 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:53:43.064053 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:53:43.064711 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:43.065206 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:43.066196 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:43.066375 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:43.066426 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:43.067225 (+ 799us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:53:43.067227 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:53:43.067227 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:53:43.067238 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:53:43.067254 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:53:43.067258 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:53:43.067301 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0806 22:53:43.067420 (+ 119us) server_negotiation.cc:299] Negotiation successful
0806 22:53:43.067462 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260806 22:53:43.067726 29547 external_mini_cluster.cc:1428] Setting key ac7bf83312d3ad7c7e957e3d82959124
I20260806 22:53:43.067761 17168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:43.063197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:41181 (local address 127.28.218.194:36429)
0806 22:53:43.063386 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0806 22:53:43.063425 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 22:53:43.063897 (+ 472us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:53:43.064124 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:53:43.064127 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:53:43.064134 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:53:43.064641 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:53:43.064645 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:53:43.065287 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:53:43.065287 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:53:43.066119 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:53:43.066121 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:53:43.066451 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:53:43.066452 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:53:43.066556 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:43.067027 (+ 471us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:53:43.067028 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:53:43.067054 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:53:43.067076 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:53:43.067328 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:53:43.067335 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:53:43.067339 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:53:43.067416 (+ 77us) client_negotiation.cc:240] Negotiation successful
0806 22:53:43.067533 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":87,"threads_started":1}
I20260806 22:53:43.068449 17166 heartbeater.cc:347] Connected to a master server at 127.28.218.254:41181
I20260806 22:53:43.068581 17166 heartbeater.cc:464] Registering TS with master...
I20260806 22:53:43.068764 17166 heartbeater.cc:511] Master 127.28.218.254:41181 requested a full tablet report, sending...
I20260806 22:53:43.068802 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056586788339-29547-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056586788339-29547-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056586788339-29547-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056586788339-29547-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.218.195:0
--local_ip_for_outbound_sockets=127.28.218.195
--webserver_interface=127.28.218.195
--webserver_port=0
--tserver_master_addrs=127.28.218.254:41181
--builtin_ntp_servers=127.28.218.212:44869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 22:53:43.069157 16827 ts_manager.cc:194] Registered new tserver with Master: 1f33d2fff7834cc3a621806f49ef1296 (127.28.218.194:36979)
I20260806 22:53:43.069547 16827 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.28.218.194:36429
W20260806 22:53:43.149060 17169 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:53:43.149299 17169 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:53:43.149343 17169 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:53:43.150569 17169 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:53:43.150614 17169 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:53:43.150674 17169 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.218.195
I20260806 22:53:43.152061 17169 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:44869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056586788339-29547-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056586788339-29547-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.218.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056586788339-29547-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.218.195
--webserver_port=0
--tserver_master_addrs=127.28.218.254:41181
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17169
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.218.195
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056586788339-29547-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:53:43.152249 17169 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:53:43.152478 17169 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:53:43.157171 17175 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:53:43.157162 17177 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:53:43.157162 17174 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:53:43.157413 17169 server_base.cc:1034] running on GCE node
I20260806 22:53:43.157651 17169 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:53:43.157878 17169 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:53:43.159039 17169 hybrid_clock.cc:648] HybridClock initialized: now 1786056823159086 us; error 88 us; skew 500 ppm
I20260806 22:53:43.160401 17169 webserver.cc:460] Webserver started at http://127.28.218.195:41085/ using document root <none> and password file <none>
I20260806 22:53:43.160663 17169 fs_manager.cc:359] Metadata directory not provided
I20260806 22:53:43.160739 17169 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:53:43.160898 17169 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:53:43.161947 17169 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056586788339-29547-0/minicluster-data/ts-2/data/instance:
uuid: "0f5717e842ac40a5a0ba12191497298c"
format_stamp: "Formatted at 2026-08-06 22:53:43 on dist-test-slave-1xrh"
server_key: "fcd5719067bbdab6cfae97d79ecb2d58"
server_key_iv: "5e6d4ea311a9dcdb74b32a3babb8c5bf"
server_key_version: "encryptionkey@0"
I20260806 22:53:43.162328 17169 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056586788339-29547-0/minicluster-data/ts-2/wal/instance:
uuid: "0f5717e842ac40a5a0ba12191497298c"
format_stamp: "Formatted at 2026-08-06 22:53:43 on dist-test-slave-1xrh"
server_key: "fcd5719067bbdab6cfae97d79ecb2d58"
server_key_iv: "5e6d4ea311a9dcdb74b32a3babb8c5bf"
server_key_version: "encryptionkey@0"
I20260806 22:53:43.163818 17169 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:53:43.164551 17183 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:53:43.164732 17169 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:53:43.164804 17169 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056586788339-29547-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056586788339-29547-0/minicluster-data/ts-2/wal
uuid: "0f5717e842ac40a5a0ba12191497298c"
format_stamp: "Formatted at 2026-08-06 22:53:43 on dist-test-slave-1xrh"
server_key: "fcd5719067bbdab6cfae97d79ecb2d58"
server_key_iv: "5e6d4ea311a9dcdb74b32a3babb8c5bf"
server_key_version: "encryptionkey@0"
I20260806 22:53:43.164891 17169 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056586788339-29547-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056586788339-29547-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056586788339-29547-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:53:43.180033 17169 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:53:43.180303 17169 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:53:43.180451 17169 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:53:43.180665 17169 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:53:43.180979 17169 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:53:43.181012 17169 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:53:43.181034 17169 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:53:43.181102 17169 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:53:43.186792 17169 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.195:40961
I20260806 22:53:43.186833 17300 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.195:40961 every 8 connection(s)
I20260806 22:53:43.187176 17169 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056586788339-29547-0/minicluster-data/ts-2/data/info.pb
I20260806 22:53:43.192319 16868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:43.187873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.195:47589 (local address 127.28.218.254:41181)
0806 22:53:43.187960 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0806 22:53:43.187963 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:53:43.188169 (+ 206us) server_negotiation.cc:407] Connection header received
0806 22:53:43.188601 (+ 432us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:53:43.188602 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:53:43.188613 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:53:43.188660 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:53:43.189287 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:43.189770 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:43.190772 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:43.191033 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:43.191105 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:43.192032 (+ 927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:53:43.192033 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:53:43.192033 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:53:43.192038 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:53:43.192054 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:53:43.192058 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:53:43.192140 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 22:53:43.192236 (+ 96us) server_negotiation.cc:299] Negotiation successful
0806 22:53:43.192260 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 22:53:43.192466 17303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:43.187842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:41181 (local address 127.28.218.195:47589)
0806 22:53:43.188052 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0806 22:53:43.188083 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0806 22:53:43.188523 (+ 440us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:53:43.188714 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:53:43.188717 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:53:43.188724 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:53:43.189210 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:53:43.189214 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:53:43.189928 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:53:43.189929 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:53:43.190697 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:53:43.190699 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:53:43.191170 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:53:43.191171 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:53:43.191299 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:43.191859 (+ 560us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:53:43.191860 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:53:43.191886 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:53:43.191908 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:53:43.192161 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:53:43.192164 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:53:43.192166 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 22:53:43.192214 (+ 48us) client_negotiation.cc:240] Negotiation successful
0806 22:53:43.192284 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":109,"threads_started":1}
I20260806 22:53:43.192977 17301 heartbeater.cc:347] Connected to a master server at 127.28.218.254:41181
I20260806 22:53:43.193094 17301 heartbeater.cc:464] Registering TS with master...
I20260806 22:53:43.193277 17301 heartbeater.cc:511] Master 127.28.218.254:41181 requested a full tablet report, sending...
I20260806 22:53:43.193722 16827 ts_manager.cc:194] Registered new tserver with Master: 0f5717e842ac40a5a0ba12191497298c (127.28.218.195:40961)
I20260806 22:53:43.194051 16827 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.28.218.195:47589
I20260806 22:53:43.194276 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 17169
I20260806 22:53:43.194391 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056586788339-29547-0/minicluster-data/ts-2/wal/instance
I20260806 22:53:43.194566 29547 external_mini_cluster.cc:1428] Setting key d6ff5bba4d91f09ce584bdfdb4e10772
I20260806 22:53:43.195508 29547 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:53:43.199466 16868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:43.196793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40616 (local address 127.28.218.254:41181)
0806 22:53:43.196826 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0806 22:53:43.196827 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:53:43.196978 (+ 151us) server_negotiation.cc:407] Connection header received
0806 22:53:43.197004 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:53:43.197005 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:53:43.197013 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:53:43.197080 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:53:43.197194 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:43.197624 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:43.198314 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:43.198485 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:43.198536 (+ 51us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:43.199225 (+ 689us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:53:43.199225 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:53:43.199226 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:53:43.199229 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:53:43.199240 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:53:43.199242 (+ 2us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:53:43.199302 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 22:53:43.199348 (+ 46us) server_negotiation.cc:299] Negotiation successful
0806 22:53:43.199356 (+ 8us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 22:53:43.201062 16827 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40616:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:53:43.202622 16827 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:53:43.209295 17311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:43.204842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35634 (local address 127.28.218.193:42603)
0806 22:53:43.205576 (+ 734us) server_negotiation.cc:206] Beginning negotiation
0806 22:53:43.205581 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:53:43.205590 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:53:43.205631 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:53:43.205633 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:53:43.205677 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:53:43.205747 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:53:43.205939 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:43.207005 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:43.208171 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:43.208909 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:43.209005 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:43.209102 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 22:53:43.209163 (+ 61us) server_negotiation.cc:299] Negotiation successful
0806 22:53:43.209213 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":687,"thread_start_us":99,"threads_started":1}
I20260806 22:53:43.209296 17313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:43.204731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.193:42603 (local address 127.0.0.1:35634)
0806 22:53:43.205060 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0806 22:53:43.205092 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0806 22:53:43.205160 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:53:43.205801 (+ 641us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:53:43.205802 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:53:43.205804 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:53:43.205845 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:53:43.205846 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:53:43.207094 (+ 1248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:53:43.207096 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:53:43.208077 (+ 981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:53:43.208080 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:53:43.209000 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:53:43.209001 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:53:43.209055 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:43.209065 (+ 10us) client_negotiation.cc:769] Sending connection context
0806 22:53:43.209152 (+ 87us) client_negotiation.cc:240] Negotiation successful
0806 22:53:43.209216 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":314,"mutex_wait_us":4,"spinlock_wait_cycles":896,"thread_start_us":91,"threads_started":1}
I20260806 22:53:43.209826 17314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:43.205044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38228 (local address 127.28.218.194:36979)
0806 22:53:43.205248 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0806 22:53:43.205253 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:53:43.205262 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:53:43.205304 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:53:43.205305 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:53:43.205316 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:53:43.205425 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:53:43.205632 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:43.206775 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:43.208358 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:43.209047 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:43.209120 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:43.209208 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 22:53:43.209708 (+ 500us) server_negotiation.cc:299] Negotiation successful
0806 22:53:43.209749 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":96,"threads_started":1}
I20260806 22:53:43.210050 17310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:43.204935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.194:36979 (local address 127.0.0.1:38228)
0806 22:53:43.205076 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0806 22:53:43.205110 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0806 22:53:43.205163 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:53:43.205429 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:53:43.205431 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:53:43.205433 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:53:43.205493 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:53:43.205495 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:53:43.206873 (+ 1378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:53:43.206874 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:53:43.208257 (+ 1383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:53:43.208261 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:53:43.209129 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:53:43.209130 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:53:43.209184 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:43.209187 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:53:43.209252 (+ 65us) client_negotiation.cc:240] Negotiation successful
0806 22:53:43.209282 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"mutex_wait_us":36,"thread_start_us":85,"threads_started":1}
I20260806 22:53:43.210019 16957 tablet_service.cc:1467] Processing CreateTablet for tablet e3cbdca3716344a9997706834b40e23d (DEFAULT_TABLE table=test-table [id=72687482881a48039b2e8836b7ba1e92]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:53:43.210399 16957 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e3cbdca3716344a9997706834b40e23d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:53:43.210436 17309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:43.204731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.195:40961 (local address 127.0.0.1:58006)
0806 22:53:43.205051 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0806 22:53:43.205092 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 22:53:43.205160 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:53:43.206015 (+ 855us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:53:43.206015 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:53:43.206017 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:53:43.206052 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:53:43.206053 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:53:43.206902 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:53:43.206903 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:53:43.208273 (+ 1370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:53:43.208275 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:53:43.210165 (+ 1890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:53:43.210166 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:53:43.210259 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:43.210263 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:53:43.210322 (+ 59us) client_negotiation.cc:240] Negotiation successful
0806 22:53:43.210362 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"mutex_wait_us":55,"thread_start_us":65,"threads_started":1}
I20260806 22:53:43.210440 17312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:43.204901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58006 (local address 127.28.218.195:40961)
0806 22:53:43.205869 (+ 968us) server_negotiation.cc:206] Beginning negotiation
0806 22:53:43.205873 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:53:43.205883 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:53:43.205915 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:53:43.205917 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:53:43.205930 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:53:43.206000 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:53:43.206104 (+ 104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:43.206784 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:43.209243 (+ 2459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:43.209986 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:43.210054 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:43.210130 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 22:53:43.210336 (+ 206us) server_negotiation.cc:299] Negotiation successful
0806 22:53:43.210368 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":934,"thread_start_us":65,"threads_started":1}
I20260806 22:53:43.210856 17096 tablet_service.cc:1467] Processing CreateTablet for tablet e3cbdca3716344a9997706834b40e23d (DEFAULT_TABLE table=test-table [id=72687482881a48039b2e8836b7ba1e92]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:53:43.211228 17096 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e3cbdca3716344a9997706834b40e23d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:53:43.211258 17233 tablet_service.cc:1467] Processing CreateTablet for tablet e3cbdca3716344a9997706834b40e23d (DEFAULT_TABLE table=test-table [id=72687482881a48039b2e8836b7ba1e92]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:53:43.211613 17233 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e3cbdca3716344a9997706834b40e23d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:53:43.213907 17315 tablet_bootstrap.cc:492] T e3cbdca3716344a9997706834b40e23d P 4f3687580aff44ca991bebd065e20d15: Bootstrap starting.
I20260806 22:53:43.214197 17316 tablet_bootstrap.cc:492] T e3cbdca3716344a9997706834b40e23d P 0f5717e842ac40a5a0ba12191497298c: Bootstrap starting.
I20260806 22:53:43.214542 17318 tablet_bootstrap.cc:492] T e3cbdca3716344a9997706834b40e23d P 1f33d2fff7834cc3a621806f49ef1296: Bootstrap starting.
I20260806 22:53:43.214740 17315 tablet_bootstrap.cc:654] T e3cbdca3716344a9997706834b40e23d P 4f3687580aff44ca991bebd065e20d15: Neither blocks nor log segments found. Creating new log.
I20260806 22:53:43.214838 17316 tablet_bootstrap.cc:654] T e3cbdca3716344a9997706834b40e23d P 0f5717e842ac40a5a0ba12191497298c: Neither blocks nor log segments found. Creating new log.
I20260806 22:53:43.215173 17315 log.cc:824] T e3cbdca3716344a9997706834b40e23d P 4f3687580aff44ca991bebd065e20d15: Log is configured to *not* fsync() on all Append() calls
I20260806 22:53:43.215207 17318 tablet_bootstrap.cc:654] T e3cbdca3716344a9997706834b40e23d P 1f33d2fff7834cc3a621806f49ef1296: Neither blocks nor log segments found. Creating new log.
I20260806 22:53:43.215220 17316 log.cc:824] T e3cbdca3716344a9997706834b40e23d P 0f5717e842ac40a5a0ba12191497298c: Log is configured to *not* fsync() on all Append() calls
I20260806 22:53:43.215533 17318 log.cc:824] T e3cbdca3716344a9997706834b40e23d P 1f33d2fff7834cc3a621806f49ef1296: Log is configured to *not* fsync() on all Append() calls
I20260806 22:53:43.215965 17316 tablet_bootstrap.cc:492] T e3cbdca3716344a9997706834b40e23d P 0f5717e842ac40a5a0ba12191497298c: No bootstrap required, opened a new log
I20260806 22:53:43.215987 17315 tablet_bootstrap.cc:492] T e3cbdca3716344a9997706834b40e23d P 4f3687580aff44ca991bebd065e20d15: No bootstrap required, opened a new log
I20260806 22:53:43.216037 17316 ts_tablet_manager.cc:1397] T e3cbdca3716344a9997706834b40e23d P 0f5717e842ac40a5a0ba12191497298c: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260806 22:53:43.216053 17315 ts_tablet_manager.cc:1397] T e3cbdca3716344a9997706834b40e23d P 4f3687580aff44ca991bebd065e20d15: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:53:43.216260 17318 tablet_bootstrap.cc:492] T e3cbdca3716344a9997706834b40e23d P 1f33d2fff7834cc3a621806f49ef1296: No bootstrap required, opened a new log
I20260806 22:53:43.216320 17318 ts_tablet_manager.cc:1397] T e3cbdca3716344a9997706834b40e23d P 1f33d2fff7834cc3a621806f49ef1296: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260806 22:53:43.218051 17315 raft_consensus.cc:348] T e3cbdca3716344a9997706834b40e23d P 4f3687580aff44ca991bebd065e20d15 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f5717e842ac40a5a0ba12191497298c" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 40961 } } peers { permanent_uuid: "4f3687580aff44ca991bebd065e20d15" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 42603 } } peers { permanent_uuid: "1f33d2fff7834cc3a621806f49ef1296" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 36979 } }
I20260806 22:53:43.218209 17315 raft_consensus.cc:374] T e3cbdca3716344a9997706834b40e23d P 4f3687580aff44ca991bebd065e20d15 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:53:43.218256 17315 raft_consensus.cc:729] T e3cbdca3716344a9997706834b40e23d P 4f3687580aff44ca991bebd065e20d15 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f3687580aff44ca991bebd065e20d15, State: Initialized, Role: FOLLOWER
I20260806 22:53:43.218369 17315 consensus_queue.cc:260] T e3cbdca3716344a9997706834b40e23d P 4f3687580aff44ca991bebd065e20d15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f5717e842ac40a5a0ba12191497298c" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 40961 } } peers { permanent_uuid: "4f3687580aff44ca991bebd065e20d15" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 42603 } } peers { permanent_uuid: "1f33d2fff7834cc3a621806f49ef1296" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 36979 } }
I20260806 22:53:43.218688 17315 ts_tablet_manager.cc:1428] T e3cbdca3716344a9997706834b40e23d P 4f3687580aff44ca991bebd065e20d15: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260806 22:53:43.218706 17026 heartbeater.cc:503] Master 127.28.218.254:41181 was elected leader, sending a full tablet report...
I20260806 22:53:43.218644 17316 raft_consensus.cc:348] T e3cbdca3716344a9997706834b40e23d P 0f5717e842ac40a5a0ba12191497298c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f5717e842ac40a5a0ba12191497298c" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 40961 } } peers { permanent_uuid: "4f3687580aff44ca991bebd065e20d15" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 42603 } } peers { permanent_uuid: "1f33d2fff7834cc3a621806f49ef1296" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 36979 } }
I20260806 22:53:43.218814 17316 raft_consensus.cc:374] T e3cbdca3716344a9997706834b40e23d P 0f5717e842ac40a5a0ba12191497298c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:53:43.218864 17316 raft_consensus.cc:729] T e3cbdca3716344a9997706834b40e23d P 0f5717e842ac40a5a0ba12191497298c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f5717e842ac40a5a0ba12191497298c, State: Initialized, Role: FOLLOWER
I20260806 22:53:43.218811 17318 raft_consensus.cc:348] T e3cbdca3716344a9997706834b40e23d P 1f33d2fff7834cc3a621806f49ef1296 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f5717e842ac40a5a0ba12191497298c" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 40961 } } peers { permanent_uuid: "4f3687580aff44ca991bebd065e20d15" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 42603 } } peers { permanent_uuid: "1f33d2fff7834cc3a621806f49ef1296" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 36979 } }
I20260806 22:53:43.218947 17318 raft_consensus.cc:374] T e3cbdca3716344a9997706834b40e23d P 1f33d2fff7834cc3a621806f49ef1296 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:53:43.219003 17318 raft_consensus.cc:729] T e3cbdca3716344a9997706834b40e23d P 1f33d2fff7834cc3a621806f49ef1296 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f33d2fff7834cc3a621806f49ef1296, State: Initialized, Role: FOLLOWER
I20260806 22:53:43.219024 17316 consensus_queue.cc:260] T e3cbdca3716344a9997706834b40e23d P 0f5717e842ac40a5a0ba12191497298c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f5717e842ac40a5a0ba12191497298c" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 40961 } } peers { permanent_uuid: "4f3687580aff44ca991bebd065e20d15" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 42603 } } peers { permanent_uuid: "1f33d2fff7834cc3a621806f49ef1296" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 36979 } }
I20260806 22:53:43.219158 17318 consensus_queue.cc:260] T e3cbdca3716344a9997706834b40e23d P 1f33d2fff7834cc3a621806f49ef1296 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f5717e842ac40a5a0ba12191497298c" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 40961 } } peers { permanent_uuid: "4f3687580aff44ca991bebd065e20d15" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 42603 } } peers { permanent_uuid: "1f33d2fff7834cc3a621806f49ef1296" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 36979 } }
I20260806 22:53:43.219301 17316 ts_tablet_manager.cc:1428] T e3cbdca3716344a9997706834b40e23d P 0f5717e842ac40a5a0ba12191497298c: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260806 22:53:43.219354 17301 heartbeater.cc:503] Master 127.28.218.254:41181 was elected leader, sending a full tablet report...
I20260806 22:53:43.219399 17318 ts_tablet_manager.cc:1428] T e3cbdca3716344a9997706834b40e23d P 1f33d2fff7834cc3a621806f49ef1296: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260806 22:53:43.219503 17166 heartbeater.cc:503] Master 127.28.218.254:41181 was elected leader, sending a full tablet report...
W20260806 22:53:43.315824 17167 tablet.cc:2369] T e3cbdca3716344a9997706834b40e23d P 1f33d2fff7834cc3a621806f49ef1296: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 22:53:43.437692 17302 tablet.cc:2369] T e3cbdca3716344a9997706834b40e23d P 0f5717e842ac40a5a0ba12191497298c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 22:53:43.437996 17027 tablet.cc:2369] T e3cbdca3716344a9997706834b40e23d P 4f3687580aff44ca991bebd065e20d15: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:53:43.510118 17324 raft_consensus.cc:482] T e3cbdca3716344a9997706834b40e23d P 1f33d2fff7834cc3a621806f49ef1296 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:53:43.510251 17324 raft_consensus.cc:504] T e3cbdca3716344a9997706834b40e23d P 1f33d2fff7834cc3a621806f49ef1296 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f5717e842ac40a5a0ba12191497298c" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 40961 } } peers { permanent_uuid: "4f3687580aff44ca991bebd065e20d15" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 42603 } } peers { permanent_uuid: "1f33d2fff7834cc3a621806f49ef1296" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 36979 } }
I20260806 22:53:43.510617 17324 leader_election.cc:288] T e3cbdca3716344a9997706834b40e23d P 1f33d2fff7834cc3a621806f49ef1296 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0f5717e842ac40a5a0ba12191497298c (127.28.218.195:40961), 4f3687580aff44ca991bebd065e20d15 (127.28.218.193:42603)
I20260806 22:53:43.513417 17311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:43.510876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.194:34573 (local address 127.28.218.193:42603)
0806 22:53:43.510921 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 22:53:43.510923 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:53:43.510937 (+ 14us) server_negotiation.cc:407] Connection header received
0806 22:53:43.510975 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:53:43.510975 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:53:43.510998 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:53:43.511098 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:53:43.511287 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:43.511758 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:43.512697 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:43.513061 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:43.513155 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:43.513164 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 22:53:43.513307 (+ 143us) server_negotiation.cc:299] Negotiation successful
0806 22:53:43.513345 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 22:53:43.513417 17168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:43.510794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.193:42603 (local address 127.28.218.194:34573)
0806 22:53:43.510849 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0806 22:53:43.510862 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:53:43.510912 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:53:43.511103 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:53:43.511104 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:53:43.511106 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:53:43.511205 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:53:43.511207 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:53:43.511851 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:53:43.511853 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:53:43.512597 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:53:43.512599 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:53:43.513153 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:53:43.513154 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:53:43.513229 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:43.513232 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:53:43.513301 (+ 69us) client_negotiation.cc:240] Negotiation successful
0806 22:53:43.513344 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34,"mutex_wait_us":1}
I20260806 22:53:43.513581 17312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:43.510917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.194:57301 (local address 127.28.218.195:40961)
0806 22:53:43.510965 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 22:53:43.510967 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:53:43.511113 (+ 146us) server_negotiation.cc:407] Connection header received
0806 22:53:43.511213 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:53:43.511214 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:53:43.511223 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:53:43.511302 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:53:43.511457 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:43.511921 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:43.512850 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:43.513211 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:43.513302 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:43.513309 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0806 22:53:43.513480 (+ 171us) server_negotiation.cc:299] Negotiation successful
0806 22:53:43.513521 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 22:53:43.513590 17326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:43.510797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.195:40961 (local address 127.28.218.194:57301)
0806 22:53:43.511042 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0806 22:53:43.511051 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:53:43.511124 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:53:43.511307 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:53:43.511308 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:53:43.511309 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:53:43.511384 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:53:43.511385 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:53:43.512019 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:53:43.512020 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:53:43.512756 (+ 736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:53:43.512757 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:53:43.513305 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:53:43.513306 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:53:43.513395 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:43.513398 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:53:43.513467 (+ 69us) client_negotiation.cc:240] Negotiation successful
0806 22:53:43.513506 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":232,"mutex_wait_us":44,"thread_start_us":100,"threads_started":1}
I20260806 22:53:43.513779 16977 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3cbdca3716344a9997706834b40e23d" candidate_uuid: "1f33d2fff7834cc3a621806f49ef1296" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f3687580aff44ca991bebd065e20d15" is_pre_election: true
I20260806 22:53:43.513938 16977 raft_consensus.cc:2454] T e3cbdca3716344a9997706834b40e23d P 4f3687580aff44ca991bebd065e20d15 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f33d2fff7834cc3a621806f49ef1296 in term 0.
I20260806 22:53:43.514006 17255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3cbdca3716344a9997706834b40e23d" candidate_uuid: "1f33d2fff7834cc3a621806f49ef1296" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f5717e842ac40a5a0ba12191497298c" is_pre_election: true
I20260806 22:53:43.514196 17255 raft_consensus.cc:2454] T e3cbdca3716344a9997706834b40e23d P 0f5717e842ac40a5a0ba12191497298c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f33d2fff7834cc3a621806f49ef1296 in term 0.
I20260806 22:53:43.514251 17046 leader_election.cc:302] T e3cbdca3716344a9997706834b40e23d P 1f33d2fff7834cc3a621806f49ef1296 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f33d2fff7834cc3a621806f49ef1296, 4f3687580aff44ca991bebd065e20d15; no voters:
I20260806 22:53:43.514400 17324 raft_consensus.cc:2781] T e3cbdca3716344a9997706834b40e23d P 1f33d2fff7834cc3a621806f49ef1296 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:53:43.514469 17324 raft_consensus.cc:482] T e3cbdca3716344a9997706834b40e23d P 1f33d2fff7834cc3a621806f49ef1296 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:53:43.514500 17324 raft_consensus.cc:3037] T e3cbdca3716344a9997706834b40e23d P 1f33d2fff7834cc3a621806f49ef1296 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:53:43.515259 17324 raft_consensus.cc:504] T e3cbdca3716344a9997706834b40e23d P 1f33d2fff7834cc3a621806f49ef1296 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f5717e842ac40a5a0ba12191497298c" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 40961 } } peers { permanent_uuid: "4f3687580aff44ca991bebd065e20d15" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 42603 } } peers { permanent_uuid: "1f33d2fff7834cc3a621806f49ef1296" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 36979 } }
I20260806 22:53:43.515410 17324 leader_election.cc:288] T e3cbdca3716344a9997706834b40e23d P 1f33d2fff7834cc3a621806f49ef1296 [CANDIDATE]: Term 1 election: Requested vote from peers 0f5717e842ac40a5a0ba12191497298c (127.28.218.195:40961), 4f3687580aff44ca991bebd065e20d15 (127.28.218.193:42603)
I20260806 22:53:43.515560 16977 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3cbdca3716344a9997706834b40e23d" candidate_uuid: "1f33d2fff7834cc3a621806f49ef1296" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f3687580aff44ca991bebd065e20d15"
I20260806 22:53:43.515594 17255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3cbdca3716344a9997706834b40e23d" candidate_uuid: "1f33d2fff7834cc3a621806f49ef1296" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f5717e842ac40a5a0ba12191497298c"
I20260806 22:53:43.515658 17255 raft_consensus.cc:3037] T e3cbdca3716344a9997706834b40e23d P 0f5717e842ac40a5a0ba12191497298c [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:53:43.515663 16977 raft_consensus.cc:3037] T e3cbdca3716344a9997706834b40e23d P 4f3687580aff44ca991bebd065e20d15 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:53:43.516378 17255 raft_consensus.cc:2454] T e3cbdca3716344a9997706834b40e23d P 0f5717e842ac40a5a0ba12191497298c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f33d2fff7834cc3a621806f49ef1296 in term 1.
I20260806 22:53:43.516402 16977 raft_consensus.cc:2454] T e3cbdca3716344a9997706834b40e23d P 4f3687580aff44ca991bebd065e20d15 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f33d2fff7834cc3a621806f49ef1296 in term 1.
I20260806 22:53:43.516563 17046 leader_election.cc:302] T e3cbdca3716344a9997706834b40e23d P 1f33d2fff7834cc3a621806f49ef1296 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f33d2fff7834cc3a621806f49ef1296, 4f3687580aff44ca991bebd065e20d15; no voters:
I20260806 22:53:43.516681 17324 raft_consensus.cc:2781] T e3cbdca3716344a9997706834b40e23d P 1f33d2fff7834cc3a621806f49ef1296 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:53:43.516819 17324 raft_consensus.cc:686] T e3cbdca3716344a9997706834b40e23d P 1f33d2fff7834cc3a621806f49ef1296 [term 1 LEADER]: Becoming Leader. State: Replica: 1f33d2fff7834cc3a621806f49ef1296, State: Running, Role: LEADER
I20260806 22:53:43.516913 17324 consensus_queue.cc:237] T e3cbdca3716344a9997706834b40e23d P 1f33d2fff7834cc3a621806f49ef1296 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f5717e842ac40a5a0ba12191497298c" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 40961 } } peers { permanent_uuid: "4f3687580aff44ca991bebd065e20d15" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 42603 } } peers { permanent_uuid: "1f33d2fff7834cc3a621806f49ef1296" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 36979 } }
I20260806 22:53:43.517830 16827 catalog_manager.cc:5495] T e3cbdca3716344a9997706834b40e23d P 1f33d2fff7834cc3a621806f49ef1296 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1f33d2fff7834cc3a621806f49ef1296 (127.28.218.194). New cstate: current_term: 1 leader_uuid: "1f33d2fff7834cc3a621806f49ef1296" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f5717e842ac40a5a0ba12191497298c" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 40961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f3687580aff44ca991bebd065e20d15" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 42603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1f33d2fff7834cc3a621806f49ef1296" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 36979 } health_report { overall_health: HEALTHY } } }
I20260806 22:53:43.560997 17314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:43.558790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38244 (local address 127.28.218.194:36979)
0806 22:53:43.558851 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0806 22:53:43.558853 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:53:43.558861 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:53:43.558917 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:53:43.558918 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:53:43.558938 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:53:43.558998 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:53:43.559148 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:43.559606 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:43.560297 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:43.560483 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:43.560530 (+ 47us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:43.560659 (+ 129us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:53:43.560740 (+ 81us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:53:43.560809 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 22:53:43.560853 (+ 44us) server_negotiation.cc:299] Negotiation successful
0806 22:53:43.560861 (+ 8us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 22:53:43.562811 16977 raft_consensus.cc:1261] T e3cbdca3716344a9997706834b40e23d P 4f3687580aff44ca991bebd065e20d15 [term 1 FOLLOWER]: Refusing update from remote peer 1f33d2fff7834cc3a621806f49ef1296: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:53:43.562811 17255 raft_consensus.cc:1261] T e3cbdca3716344a9997706834b40e23d P 0f5717e842ac40a5a0ba12191497298c [term 1 FOLLOWER]: Refusing update from remote peer 1f33d2fff7834cc3a621806f49ef1296: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:53:43.563199 17324 consensus_queue.cc:1035] T e3cbdca3716344a9997706834b40e23d P 1f33d2fff7834cc3a621806f49ef1296 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f5717e842ac40a5a0ba12191497298c" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 40961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:53:43.563365 17327 consensus_queue.cc:1035] T e3cbdca3716344a9997706834b40e23d P 1f33d2fff7834cc3a621806f49ef1296 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f3687580aff44ca991bebd065e20d15" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 42603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:53:43.565708 17330 mvcc.cc:205] Tried to move back new op lower bound from 7315688749310603264 to 7315688749125902336. Current Snapshot: MvccSnapshot[applied={T|T < 7315688749310603264}]
I20260806 22:53:43.566437 17332 mvcc.cc:205] Tried to move back new op lower bound from 7315688749310603264 to 7315688749125902336. Current Snapshot: MvccSnapshot[applied={T|T < 7315688749310603264 or (T in {7315688749310603264})}]
I20260806 22:53:43.567044 17331 mvcc.cc:205] Tried to move back new op lower bound from 7315688749310603264 to 7315688749125902336. Current Snapshot: MvccSnapshot[applied={T|T < 7315688749310603264}]
I20260806 22:53:43.568140 16826 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:40616:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:53:43.568255 16826 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:40616:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:53:43.569079 16826 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5dddfc4701414b10acab77047a97cf61: Sending DeleteTablet for 3 replicas of tablet e3cbdca3716344a9997706834b40e23d
I20260806 22:53:43.569415 16957 tablet_service.cc:1514] Processing DeleteTablet for tablet e3cbdca3716344a9997706834b40e23d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:53:43 UTC) from {username='slave'} at 127.0.0.1:35634
I20260806 22:53:43.569439 17233 tablet_service.cc:1514] Processing DeleteTablet for tablet e3cbdca3716344a9997706834b40e23d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:53:43 UTC) from {username='slave'} at 127.0.0.1:58006
I20260806 22:53:43.569584 17096 tablet_service.cc:1514] Processing DeleteTablet for tablet e3cbdca3716344a9997706834b40e23d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:53:43 UTC) from {username='slave'} at 127.0.0.1:38228
I20260806 22:53:43.569654 17339 tablet_replica.cc:330] T e3cbdca3716344a9997706834b40e23d P 4f3687580aff44ca991bebd065e20d15: stopping tablet replica
I20260806 22:53:43.569746 17339 raft_consensus.cc:2229] T e3cbdca3716344a9997706834b40e23d P 4f3687580aff44ca991bebd065e20d15 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:53:43.569860 17339 raft_consensus.cc:2258] T e3cbdca3716344a9997706834b40e23d P 4f3687580aff44ca991bebd065e20d15 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:53:43.569917 17340 tablet_replica.cc:330] T e3cbdca3716344a9997706834b40e23d P 0f5717e842ac40a5a0ba12191497298c: stopping tablet replica
I20260806 22:53:43.570024 17340 raft_consensus.cc:2229] T e3cbdca3716344a9997706834b40e23d P 0f5717e842ac40a5a0ba12191497298c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:53:43.570104 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 16889
I20260806 22:53:43.570128 17340 raft_consensus.cc:2258] T e3cbdca3716344a9997706834b40e23d P 0f5717e842ac40a5a0ba12191497298c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:53:43.570267 17341 tablet_replica.cc:330] T e3cbdca3716344a9997706834b40e23d P 1f33d2fff7834cc3a621806f49ef1296: stopping tablet replica
I20260806 22:53:43.570349 17341 raft_consensus.cc:2229] T e3cbdca3716344a9997706834b40e23d P 1f33d2fff7834cc3a621806f49ef1296 [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:53:43.570433 17341 raft_consensus.cc:2258] T e3cbdca3716344a9997706834b40e23d P 1f33d2fff7834cc3a621806f49ef1296 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:53:43.570907 17341 ts_tablet_manager.cc:1905] T e3cbdca3716344a9997706834b40e23d P 1f33d2fff7834cc3a621806f49ef1296: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:53:43.571535 17340 ts_tablet_manager.cc:1905] T e3cbdca3716344a9997706834b40e23d P 0f5717e842ac40a5a0ba12191497298c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:53:43.572976 17341 ts_tablet_manager.cc:1918] T e3cbdca3716344a9997706834b40e23d P 1f33d2fff7834cc3a621806f49ef1296: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:53:43.573067 17341 log.cc:1196] T e3cbdca3716344a9997706834b40e23d P 1f33d2fff7834cc3a621806f49ef1296: Deleting WAL directory at /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056586788339-29547-0/minicluster-data/ts-1/wal/wals/e3cbdca3716344a9997706834b40e23d
I20260806 22:53:43.573304 17340 ts_tablet_manager.cc:1918] T e3cbdca3716344a9997706834b40e23d P 0f5717e842ac40a5a0ba12191497298c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:53:43.573354 17341 ts_tablet_manager.cc:1939] T e3cbdca3716344a9997706834b40e23d P 1f33d2fff7834cc3a621806f49ef1296: Deleting consensus metadata
I20260806 22:53:43.573565 17340 log.cc:1196] T e3cbdca3716344a9997706834b40e23d P 0f5717e842ac40a5a0ba12191497298c: Deleting WAL directory at /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056586788339-29547-0/minicluster-data/ts-2/wal/wals/e3cbdca3716344a9997706834b40e23d
I20260806 22:53:43.573876 17340 ts_tablet_manager.cc:1939] T e3cbdca3716344a9997706834b40e23d P 0f5717e842ac40a5a0ba12191497298c: Deleting consensus metadata
I20260806 22:53:43.574347 16811 catalog_manager.cc:4841] TS 0f5717e842ac40a5a0ba12191497298c (127.28.218.195:40961): tablet e3cbdca3716344a9997706834b40e23d (table test-table [id=72687482881a48039b2e8836b7ba1e92]) successfully deleted
I20260806 22:53:43.574676 16811 catalog_manager.cc:4841] TS 1f33d2fff7834cc3a621806f49ef1296 (127.28.218.194:36979): tablet e3cbdca3716344a9997706834b40e23d (table test-table [id=72687482881a48039b2e8836b7ba1e92]) successfully deleted
W20260806 22:53:43.578332 16812 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.28.218.193:42603 (error 108)
I20260806 22:53:43.578589 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 17029
I20260806 22:53:43.579255 17310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:43.578733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.193:42603 (local address 127.0.0.1:35644)
0806 22:53:43.579181 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0806 22:53:43.579228 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.193:42603: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":430}
W20260806 22:53:43.579397 16812 catalog_manager.cc:4584] TS 4f3687580aff44ca991bebd065e20d15 (127.28.218.193:42603): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e3cbdca3716344a9997706834b40e23d: Network error: Client connection negotiation failed: client connection to 127.28.218.193:42603: connect: Connection refused (error 111)
I20260806 22:53:43.584041 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 17169
I20260806 22:53:43.588783 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 16796
2026-08-06T22:53:43Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (958 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056586788339-29547-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:53:43 dist-test-slave-1xrh krb5kdc[17345](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:53:43 dist-test-slave-1xrh krb5kdc[17345](info): set up 2 sockets
Aug 06 22:53:43 dist-test-slave-1xrh krb5kdc[17345](info): commencing operation
krb5kdc: starting...
W20260806 22:53:44.638908 29547 mini_kdc.cc:121] Time spent starting KDC: real 1.029s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056586788339-29547-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056586788339-29547-0/krb5kdc/test-user.keytab.
Aug 06 22:53:44 dist-test-slave-1xrh krb5kdc[17345](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056824, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:53:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:53:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.218.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.218.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.218.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab.
Entry for principal kudu/127.28.218.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab.
I20260806 22:53:44.758351 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056586788339-29547-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056586788339-29547-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056586788339-29547-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.218.254:36659
--webserver_interface=127.28.218.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab
--principal=kudu/127.28.218.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.218.212:40991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.218.254:36659
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:53:44.838958 17358 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:53:44.839223 17358 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:53:44.839277 17358 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:53:44.840433 17358 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:53:44.840516 17358 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:53:44.840545 17358 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:53:44.840575 17358 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:53:44.840597 17358 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:53:44.842067 17358 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:40991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056586788339-29547-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056586788339-29547-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.218.254:36659
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab
--principal=kudu/127.28.218.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.218.254:36659
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.218.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17358
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056586788339-29547-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:53:44.842346 17358 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:53:44.842588 17358 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:53:44.847347 17367 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:53:44.847397 17358 server_base.cc:1034] running on GCE node
W20260806 22:53:44.847379 17364 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:53:44.847380 17365 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:53:44.847827 17358 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:53:44.848074 17358 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:53:44.849222 17358 hybrid_clock.cc:648] HybridClock initialized: now 1786056824849200 us; error 38 us; skew 500 ppm
Aug 06 22:53:44 dist-test-slave-1xrh krb5kdc[17345](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056824, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:53:44.851723 17358 init.cc:377] Logged in from keytab as kudu/127.28.218.254@KRBTEST.COM (short username kudu)
I20260806 22:53:44.852186 17358 webserver.cc:460] Webserver started at http://127.28.218.254:43277/ using document root <none> and password file <none>
I20260806 22:53:44.852433 17358 fs_manager.cc:359] Metadata directory not provided
I20260806 22:53:44.852478 17358 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:53:44.852674 17358 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:53:44.853768 17358 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056586788339-29547-0/minicluster-data/master-0/data/instance:
uuid: "bd39ee413b9944a896931cdbeb937d08"
format_stamp: "Formatted at 2026-08-06 22:53:44 on dist-test-slave-1xrh"
server_key: "4c2b14089e22a16fcbb8252256be1db2"
server_key_iv: "24f9ca47aa8963eb98669dcd61bb672c"
server_key_version: "encryptionkey@0"
I20260806 22:53:44.854118 17358 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056586788339-29547-0/minicluster-data/master-0/wal/instance:
uuid: "bd39ee413b9944a896931cdbeb937d08"
format_stamp: "Formatted at 2026-08-06 22:53:44 on dist-test-slave-1xrh"
server_key: "4c2b14089e22a16fcbb8252256be1db2"
server_key_iv: "24f9ca47aa8963eb98669dcd61bb672c"
server_key_version: "encryptionkey@0"
I20260806 22:53:44.855428 17358 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:53:44.856166 17374 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:53:44.856377 17358 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:53:44.856458 17358 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056586788339-29547-0/minicluster-data/master-0/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056586788339-29547-0/minicluster-data/master-0/wal
uuid: "bd39ee413b9944a896931cdbeb937d08"
format_stamp: "Formatted at 2026-08-06 22:53:44 on dist-test-slave-1xrh"
server_key: "4c2b14089e22a16fcbb8252256be1db2"
server_key_iv: "24f9ca47aa8963eb98669dcd61bb672c"
server_key_version: "encryptionkey@0"
I20260806 22:53:44.856535 17358 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056586788339-29547-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056586788339-29547-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056586788339-29547-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:53:44.868515 17358 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:53:44.870699 17358 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:53:44.870806 17358 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:53:44.874771 17358 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.254:36659
I20260806 22:53:44.874825 17428 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.254:36659 every 8 connection(s)
I20260806 22:53:44.875219 17358 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
I20260806 22:53:44.876104 17429 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:53:44.878312 17429 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd39ee413b9944a896931cdbeb937d08: Bootstrap starting.
I20260806 22:53:44.878882 17429 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bd39ee413b9944a896931cdbeb937d08: Neither blocks nor log segments found. Creating new log.
I20260806 22:53:44.879246 17429 log.cc:824] T 00000000000000000000000000000000 P bd39ee413b9944a896931cdbeb937d08: Log is configured to *not* fsync() on all Append() calls
I20260806 22:53:44.879932 17429 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd39ee413b9944a896931cdbeb937d08: No bootstrap required, opened a new log
I20260806 22:53:44.881482 17429 raft_consensus.cc:348] T 00000000000000000000000000000000 P bd39ee413b9944a896931cdbeb937d08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd39ee413b9944a896931cdbeb937d08" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 36659 } }
I20260806 22:53:44.881660 17429 raft_consensus.cc:374] T 00000000000000000000000000000000 P bd39ee413b9944a896931cdbeb937d08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:53:44.881707 17429 raft_consensus.cc:729] T 00000000000000000000000000000000 P bd39ee413b9944a896931cdbeb937d08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd39ee413b9944a896931cdbeb937d08, State: Initialized, Role: FOLLOWER
I20260806 22:53:44.881893 17429 consensus_queue.cc:260] T 00000000000000000000000000000000 P bd39ee413b9944a896931cdbeb937d08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd39ee413b9944a896931cdbeb937d08" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 36659 } }
I20260806 22:53:44.881994 17429 raft_consensus.cc:388] T 00000000000000000000000000000000 P bd39ee413b9944a896931cdbeb937d08 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:53:44.882035 17429 raft_consensus.cc:482] T 00000000000000000000000000000000 P bd39ee413b9944a896931cdbeb937d08 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:53:44.882087 17429 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bd39ee413b9944a896931cdbeb937d08 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:53:44.882608 17429 raft_consensus.cc:504] T 00000000000000000000000000000000 P bd39ee413b9944a896931cdbeb937d08 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd39ee413b9944a896931cdbeb937d08" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 36659 } }
I20260806 22:53:44.882740 17429 leader_election.cc:302] T 00000000000000000000000000000000 P bd39ee413b9944a896931cdbeb937d08 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bd39ee413b9944a896931cdbeb937d08; no voters:
I20260806 22:53:44.882917 17429 leader_election.cc:288] T 00000000000000000000000000000000 P bd39ee413b9944a896931cdbeb937d08 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:53:44.882958 17432 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bd39ee413b9944a896931cdbeb937d08 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:53:44.882988 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 17358
I20260806 22:53:44.883095 17432 raft_consensus.cc:686] T 00000000000000000000000000000000 P bd39ee413b9944a896931cdbeb937d08 [term 1 LEADER]: Becoming Leader. State: Replica: bd39ee413b9944a896931cdbeb937d08, State: Running, Role: LEADER
I20260806 22:53:44.883113 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056586788339-29547-0/minicluster-data/master-0/wal/instance
I20260806 22:53:44.883193 17432 consensus_queue.cc:237] T 00000000000000000000000000000000 P bd39ee413b9944a896931cdbeb937d08 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd39ee413b9944a896931cdbeb937d08" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 36659 } }
I20260806 22:53:44.883316 29547 external_mini_cluster.cc:1428] Setting key 66013e22b4088b45e1920f087c943798
I20260806 22:53:44.883194 17429 sys_catalog.cc:564] T 00000000000000000000000000000000 P bd39ee413b9944a896931cdbeb937d08 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:53:44.883682 17434 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd39ee413b9944a896931cdbeb937d08 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bd39ee413b9944a896931cdbeb937d08. Latest consensus state: current_term: 1 leader_uuid: "bd39ee413b9944a896931cdbeb937d08" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd39ee413b9944a896931cdbeb937d08" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 36659 } } }
I20260806 22:53:44.883764 17434 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd39ee413b9944a896931cdbeb937d08 [sys.catalog]: This master's current role is: LEADER
I20260806 22:53:44.883526 17433 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd39ee413b9944a896931cdbeb937d08 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bd39ee413b9944a896931cdbeb937d08" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd39ee413b9944a896931cdbeb937d08" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 36659 } } }
I20260806 22:53:44.884020 17433 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd39ee413b9944a896931cdbeb937d08 [sys.catalog]: This master's current role is: LEADER
I20260806 22:53:44.884356 17437 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:53:44.884872 17437 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:53:44.886615 17437 catalog_manager.cc:1347] Generated new cluster ID: 5020be89b5914392846f17293261eb6b
I20260806 22:53:44.886664 17437 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 22:53:44 dist-test-slave-1xrh krb5kdc[17345](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056824, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.218.254@KRBTEST.COM
I20260806 22:53:44.895704 17439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:44.883980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38392 (local address 127.28.218.254:36659)
0806 22:53:44.884165 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0806 22:53:44.884175 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:53:44.884397 (+ 222us) server_negotiation.cc:407] Connection header received
0806 22:53:44.884598 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:53:44.884605 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:53:44.884666 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:53:44.884810 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:53:44.885712 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:44.886890 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:44.887953 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:44.888534 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:44.888625 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:44.891454 (+ 2829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:53:44.891458 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:53:44.891458 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:53:44.891470 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:53:44.894005 (+ 2535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:53:44.894411 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:53:44.894414 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:53:44.894414 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:53:44.894546 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:53:44.894755 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:53:44.894756 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:53:44.894757 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:53:44.895079 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:53:44.895277 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0806 22:53:44.895351 (+ 74us) server_negotiation.cc:299] Negotiation successful
0806 22:53:44.895500 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":90,"threads_started":1}
I20260806 22:53:44.898216 17437 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:53:44.898662 17437 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:53:44.901901 17437 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bd39ee413b9944a896931cdbeb937d08: Generated new TSK 0
I20260806 22:53:44.902076 17437 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.28.218.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.218.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.218.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.193.keytab.
Entry for principal kudu/127.28.218.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.193.keytab.
I20260806 22:53:44.934628 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056586788339-29547-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056586788339-29547-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056586788339-29547-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056586788339-29547-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.218.193:0
--local_ip_for_outbound_sockets=127.28.218.193
--webserver_interface=127.28.218.193
--webserver_port=0
--tserver_master_addrs=127.28.218.254:36659
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.193.keytab
--principal=kudu/127.28.218.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.218.212:40991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:53:45.015904 17455 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:53:45.016139 17455 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:53:45.016176 17455 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:53:45.017338 17455 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:53:45.017410 17455 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:53:45.017454 17455 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.218.193
I20260806 22:53:45.018847 17455 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:40991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056586788339-29547-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056586788339-29547-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.193.keytab
--principal=kudu/127.28.218.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.218.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056586788339-29547-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.218.193
--webserver_port=0
--tserver_master_addrs=127.28.218.254:36659
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17455
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.218.193
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056586788339-29547-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:53:45.019111 17455 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:53:45.019351 17455 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:53:45.024238 17460 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:53:45.024237 17463 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:53:45.024324 17461 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:53:45.024531 17455 server_base.cc:1034] running on GCE node
I20260806 22:53:45.024739 17455 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:53:45.024976 17455 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:53:45.026158 17455 hybrid_clock.cc:648] HybridClock initialized: now 1786056825026140 us; error 32 us; skew 500 ppm
Aug 06 22:53:45 dist-test-slave-1xrh krb5kdc[17345](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056825, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:53:45.028728 17455 init.cc:377] Logged in from keytab as kudu/127.28.218.193@KRBTEST.COM (short username kudu)
I20260806 22:53:45.029212 17455 webserver.cc:460] Webserver started at http://127.28.218.193:35753/ using document root <none> and password file <none>
I20260806 22:53:45.029443 17455 fs_manager.cc:359] Metadata directory not provided
I20260806 22:53:45.029523 17455 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:53:45.029681 17455 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:53:45.030678 17455 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056586788339-29547-0/minicluster-data/ts-0/data/instance:
uuid: "fac9a6c2a58940cca49b33029ea8d892"
format_stamp: "Formatted at 2026-08-06 22:53:45 on dist-test-slave-1xrh"
server_key: "0ddd3120f6460adb628893c0a64fa5e0"
server_key_iv: "49c5706f08bdef52217e20e14ee1d28d"
server_key_version: "encryptionkey@0"
I20260806 22:53:45.031055 17455 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056586788339-29547-0/minicluster-data/ts-0/wal/instance:
uuid: "fac9a6c2a58940cca49b33029ea8d892"
format_stamp: "Formatted at 2026-08-06 22:53:45 on dist-test-slave-1xrh"
server_key: "0ddd3120f6460adb628893c0a64fa5e0"
server_key_iv: "49c5706f08bdef52217e20e14ee1d28d"
server_key_version: "encryptionkey@0"
I20260806 22:53:45.032366 17455 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:53:45.033085 17471 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:53:45.033254 17455 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:53:45.033356 17455 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056586788339-29547-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056586788339-29547-0/minicluster-data/ts-0/wal
uuid: "fac9a6c2a58940cca49b33029ea8d892"
format_stamp: "Formatted at 2026-08-06 22:53:45 on dist-test-slave-1xrh"
server_key: "0ddd3120f6460adb628893c0a64fa5e0"
server_key_iv: "49c5706f08bdef52217e20e14ee1d28d"
server_key_version: "encryptionkey@0"
I20260806 22:53:45.033442 17455 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056586788339-29547-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056586788339-29547-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056586788339-29547-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:53:45.046551 17455 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:53:45.048362 17455 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:53:45.048534 17455 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:53:45.048753 17455 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:53:45.049068 17455 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:53:45.049144 17455 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:53:45.049177 17455 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:53:45.049203 17455 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:53:45.055045 17455 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.193:33573
I20260806 22:53:45.055166 17588 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.193:33573 every 8 connection(s)
I20260806 22:53:45.055466 17455 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056586788339-29547-0/minicluster-data/ts-0/data/info.pb
I20260806 22:53:45.059576 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 17455
I20260806 22:53:45.059713 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056586788339-29547-0/minicluster-data/ts-0/wal/instance
I20260806 22:53:45.059926 29547 external_mini_cluster.cc:1428] Setting key 27f71b0adc6c20f148a2b9ea8c658fca
Aug 06 22:53:45 dist-test-slave-1xrh krb5kdc[17345](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056825, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.193@KRBTEST.COM for kudu/127.28.218.254@KRBTEST.COM
I20260806 22:53:45.065129 17439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:45.056229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.193:47951 (local address 127.28.218.254:36659)
0806 22:53:45.056305 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0806 22:53:45.056308 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:53:45.056503 (+ 195us) server_negotiation.cc:407] Connection header received
0806 22:53:45.056910 (+ 407us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:53:45.056911 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:53:45.056924 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:53:45.056977 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:53:45.057826 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:45.058366 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:45.059428 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:45.059761 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:45.059833 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:45.062065 (+ 2232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:53:45.062067 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:53:45.062067 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:53:45.062076 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:53:45.063578 (+ 1502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:53:45.064098 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:53:45.064099 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:53:45.064099 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:53:45.064151 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:53:45.064419 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:53:45.064419 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:53:45.064420 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:53:45.064527 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:53:45.064577 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 22:53:45.064908 (+ 331us) server_negotiation.cc:299] Negotiation successful
0806 22:53:45.065037 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 22:53:45.065348 17591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:45.056163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:36659 (local address 127.28.218.193:47951)
0806 22:53:45.056370 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0806 22:53:45.056408 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 22:53:45.056833 (+ 425us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:53:45.057025 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:53:45.057028 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:53:45.057564 (+ 536us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:53:45.057733 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:53:45.057737 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:53:45.058437 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:53:45.058437 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:53:45.059332 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:53:45.059335 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:53:45.059845 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:53:45.059846 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:53:45.060017 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:45.060582 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:53:45.060592 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:53:45.061975 (+ 1383us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:53:45.063663 (+ 1688us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:53:45.063670 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:53:45.063672 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:53:45.064023 (+ 351us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:53:45.064216 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:53:45.064217 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:53:45.064218 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:53:45.064357 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:53:45.064597 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:53:45.064604 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:53:45.064771 (+ 167us) client_negotiation.cc:769] Sending connection context
0806 22:53:45.064889 (+ 118us) client_negotiation.cc:240] Negotiation successful
0806 22:53:45.065074 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":93,"threads_started":1}
I20260806 22:53:45.065840 17589 heartbeater.cc:347] Connected to a master server at 127.28.218.254:36659
I20260806 22:53:45.065938 17589 heartbeater.cc:464] Registering TS with master...
I20260806 22:53:45.066134 17589 heartbeater.cc:511] Master 127.28.218.254:36659 requested a full tablet report, sending...
I20260806 22:53:45.066607 17392 ts_manager.cc:194] Registered new tserver with Master: fac9a6c2a58940cca49b33029ea8d892 (127.28.218.193:33573)
I20260806 22:53:45.067315 17392 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.218.193@KRBTEST.COM'} at 127.28.218.193:47951
WARNING: no policy specified for kudu/127.28.218.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.218.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.218.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.194.keytab.
Entry for principal kudu/127.28.218.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.194.keytab.
I20260806 22:53:45.085894 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056586788339-29547-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056586788339-29547-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056586788339-29547-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056586788339-29547-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.218.194:0
--local_ip_for_outbound_sockets=127.28.218.194
--webserver_interface=127.28.218.194
--webserver_port=0
--tserver_master_addrs=127.28.218.254:36659
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.194.keytab
--principal=kudu/127.28.218.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.218.212:40991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:53:45.165848 17594 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:53:45.166034 17594 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:53:45.166059 17594 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:53:45.167088 17594 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:53:45.167145 17594 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:53:45.167171 17594 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.218.194
I20260806 22:53:45.168520 17594 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:40991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056586788339-29547-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056586788339-29547-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.194.keytab
--principal=kudu/127.28.218.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.218.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056586788339-29547-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.218.194
--webserver_port=0
--tserver_master_addrs=127.28.218.254:36659
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17594
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.218.194
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056586788339-29547-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:53:45.168686 17594 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:53:45.168846 17594 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:53:45.173833 17602 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:53:45.173882 17594 server_base.cc:1034] running on GCE node
W20260806 22:53:45.173830 17600 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:53:45.173832 17599 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:53:45.174309 17594 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:53:45.174566 17594 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:53:45.175717 17594 hybrid_clock.cc:648] HybridClock initialized: now 1786056825175705 us; error 31 us; skew 500 ppm
Aug 06 22:53:45 dist-test-slave-1xrh krb5kdc[17345](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056825, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:53:45.178253 17594 init.cc:377] Logged in from keytab as kudu/127.28.218.194@KRBTEST.COM (short username kudu)
I20260806 22:53:45.178669 17594 webserver.cc:460] Webserver started at http://127.28.218.194:35395/ using document root <none> and password file <none>
I20260806 22:53:45.178884 17594 fs_manager.cc:359] Metadata directory not provided
I20260806 22:53:45.178927 17594 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:53:45.179106 17594 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:53:45.180122 17594 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056586788339-29547-0/minicluster-data/ts-1/data/instance:
uuid: "9396e51d73b94aefb87f493ab3dc061b"
format_stamp: "Formatted at 2026-08-06 22:53:45 on dist-test-slave-1xrh"
server_key: "e82aade853a4bb9f523741d369aae839"
server_key_iv: "9b2a723a9719707449eb7902a21712b7"
server_key_version: "encryptionkey@0"
I20260806 22:53:45.180464 17594 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056586788339-29547-0/minicluster-data/ts-1/wal/instance:
uuid: "9396e51d73b94aefb87f493ab3dc061b"
format_stamp: "Formatted at 2026-08-06 22:53:45 on dist-test-slave-1xrh"
server_key: "e82aade853a4bb9f523741d369aae839"
server_key_iv: "9b2a723a9719707449eb7902a21712b7"
server_key_version: "encryptionkey@0"
I20260806 22:53:45.181773 17594 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:53:45.182557 17609 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:53:45.182729 17594 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:53:45.182891 17594 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056586788339-29547-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056586788339-29547-0/minicluster-data/ts-1/wal
uuid: "9396e51d73b94aefb87f493ab3dc061b"
format_stamp: "Formatted at 2026-08-06 22:53:45 on dist-test-slave-1xrh"
server_key: "e82aade853a4bb9f523741d369aae839"
server_key_iv: "9b2a723a9719707449eb7902a21712b7"
server_key_version: "encryptionkey@0"
I20260806 22:53:45.182987 17594 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056586788339-29547-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056586788339-29547-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056586788339-29547-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:53:45.196478 17594 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:53:45.198190 17594 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:53:45.198379 17594 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:53:45.198616 17594 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:53:45.198913 17594 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:53:45.198968 17594 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:53:45.199016 17594 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:53:45.199043 17594 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:53:45.204797 17594 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.194:45091
I20260806 22:53:45.204846 17730 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.194:45091 every 8 connection(s)
I20260806 22:53:45.205145 17594 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056586788339-29547-0/minicluster-data/ts-1/data/info.pb
Aug 06 22:53:45 dist-test-slave-1xrh krb5kdc[17345](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056825, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.194@KRBTEST.COM for kudu/127.28.218.254@KRBTEST.COM
I20260806 22:53:45.210628 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 17594
I20260806 22:53:45.210749 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056586788339-29547-0/minicluster-data/ts-1/wal/instance
I20260806 22:53:45.210944 29547 external_mini_cluster.cc:1428] Setting key c20087c2798e91b5781d6bf94380c213
I20260806 22:53:45.214107 17733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:45.205729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:36659 (local address 127.28.218.194:46789)
0806 22:53:45.205914 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0806 22:53:45.205952 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 22:53:45.206416 (+ 464us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:53:45.206628 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:53:45.206631 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:53:45.207181 (+ 550us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:53:45.207361 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:53:45.207365 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:53:45.207958 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:53:45.207959 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:53:45.208650 (+ 691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:53:45.208652 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:53:45.208985 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:53:45.208986 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:53:45.209106 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:45.209511 (+ 405us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:53:45.209519 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:53:45.210926 (+ 1407us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:53:45.212671 (+ 1745us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:53:45.212677 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:53:45.212679 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:53:45.213007 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:53:45.213215 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:53:45.213216 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:53:45.213217 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:53:45.213353 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:53:45.213585 (+ 232us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:53:45.213592 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:53:45.213755 (+ 163us) client_negotiation.cc:769] Sending connection context
0806 22:53:45.213852 (+ 97us) client_negotiation.cc:240] Negotiation successful
0806 22:53:45.213963 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":116,"threads_started":1}
I20260806 22:53:45.214121 17439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:45.205768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.194:46789 (local address 127.28.218.254:36659)
0806 22:53:45.205832 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0806 22:53:45.205834 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:53:45.206035 (+ 201us) server_negotiation.cc:407] Connection header received
0806 22:53:45.206501 (+ 466us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:53:45.206502 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:53:45.206511 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:53:45.206562 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:53:45.207435 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:45.207883 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:45.208724 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:45.208891 (+ 167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:45.208937 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:45.211050 (+ 2113us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:53:45.211051 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:53:45.211052 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:53:45.211059 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:53:45.212587 (+ 1528us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:53:45.213083 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:53:45.213084 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:53:45.213084 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:53:45.213135 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:53:45.213414 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:53:45.213415 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:53:45.213415 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:53:45.213512 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:53:45.213593 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 22:53:45.213907 (+ 314us) server_negotiation.cc:299] Negotiation successful
0806 22:53:45.214005 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 22:53:45.214651 17731 heartbeater.cc:347] Connected to a master server at 127.28.218.254:36659
I20260806 22:53:45.214749 17731 heartbeater.cc:464] Registering TS with master...
I20260806 22:53:45.214924 17731 heartbeater.cc:511] Master 127.28.218.254:36659 requested a full tablet report, sending...
I20260806 22:53:45.215330 17392 ts_manager.cc:194] Registered new tserver with Master: 9396e51d73b94aefb87f493ab3dc061b (127.28.218.194:45091)
I20260806 22:53:45.215826 17392 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.218.194@KRBTEST.COM'} at 127.28.218.194:46789
WARNING: no policy specified for kudu/127.28.218.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.218.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.218.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.195.keytab.
Entry for principal kudu/127.28.218.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.195.keytab.
I20260806 22:53:45.237314 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056586788339-29547-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056586788339-29547-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056586788339-29547-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056586788339-29547-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.218.195:0
--local_ip_for_outbound_sockets=127.28.218.195
--webserver_interface=127.28.218.195
--webserver_port=0
--tserver_master_addrs=127.28.218.254:36659
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.195.keytab
--principal=kudu/127.28.218.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.218.212:40991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:53:45.320436 17736 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:53:45.320648 17736 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:53:45.320672 17736 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:53:45.321921 17736 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:53:45.321961 17736 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:53:45.321990 17736 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.218.195
I20260806 22:53:45.323494 17736 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:40991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056586788339-29547-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056586788339-29547-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.195.keytab
--principal=kudu/127.28.218.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.218.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056586788339-29547-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.218.195
--webserver_port=0
--tserver_master_addrs=127.28.218.254:36659
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17736
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.218.195
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056586788339-29547-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:53:45.323746 17736 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:53:45.324007 17736 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:53:45.329461 17745 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:53:45.329507 17741 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:53:45.329461 17742 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:53:45.329701 17736 server_base.cc:1034] running on GCE node
I20260806 22:53:45.329957 17736 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:53:45.330204 17736 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:53:45.331369 17736 hybrid_clock.cc:648] HybridClock initialized: now 1786056825331352 us; error 30 us; skew 500 ppm
Aug 06 22:53:45 dist-test-slave-1xrh krb5kdc[17345](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056825, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:53:45.334034 17736 init.cc:377] Logged in from keytab as kudu/127.28.218.195@KRBTEST.COM (short username kudu)
I20260806 22:53:45.334507 17736 webserver.cc:460] Webserver started at http://127.28.218.195:38287/ using document root <none> and password file <none>
I20260806 22:53:45.334736 17736 fs_manager.cc:359] Metadata directory not provided
I20260806 22:53:45.334806 17736 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:53:45.334964 17736 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:53:45.336098 17736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056586788339-29547-0/minicluster-data/ts-2/data/instance:
uuid: "7ecaf1d8dddd4344af263a49db100370"
format_stamp: "Formatted at 2026-08-06 22:53:45 on dist-test-slave-1xrh"
server_key: "3ea42f0edbe4ed2a9ac5a8b353e79618"
server_key_iv: "b42478284d5a22d069cbffbbe1c2de91"
server_key_version: "encryptionkey@0"
I20260806 22:53:45.336512 17736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056586788339-29547-0/minicluster-data/ts-2/wal/instance:
uuid: "7ecaf1d8dddd4344af263a49db100370"
format_stamp: "Formatted at 2026-08-06 22:53:45 on dist-test-slave-1xrh"
server_key: "3ea42f0edbe4ed2a9ac5a8b353e79618"
server_key_iv: "b42478284d5a22d069cbffbbe1c2de91"
server_key_version: "encryptionkey@0"
I20260806 22:53:45.337924 17736 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:53:45.338637 17752 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:53:45.338805 17736 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:53:45.338908 17736 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056586788339-29547-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056586788339-29547-0/minicluster-data/ts-2/wal
uuid: "7ecaf1d8dddd4344af263a49db100370"
format_stamp: "Formatted at 2026-08-06 22:53:45 on dist-test-slave-1xrh"
server_key: "3ea42f0edbe4ed2a9ac5a8b353e79618"
server_key_iv: "b42478284d5a22d069cbffbbe1c2de91"
server_key_version: "encryptionkey@0"
I20260806 22:53:45.338989 17736 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056586788339-29547-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056586788339-29547-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056586788339-29547-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:53:45.363047 17736 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:53:45.364984 17736 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:53:45.365161 17736 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:53:45.365406 17736 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:53:45.365734 17736 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:53:45.365792 17736 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:53:45.365834 17736 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:53:45.365863 17736 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:53:45.371735 17736 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.195:37837
I20260806 22:53:45.371781 17873 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.195:37837 every 8 connection(s)
I20260806 22:53:45.372108 17736 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056586788339-29547-0/minicluster-data/ts-2/data/info.pb
I20260806 22:53:45.372494 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 17736
I20260806 22:53:45.372610 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056586788339-29547-0/minicluster-data/ts-2/wal/instance
I20260806 22:53:45.372830 29547 external_mini_cluster.cc:1428] Setting key 148e0524f1cec700b0ef829979cdbc32
Aug 06 22:53:45 dist-test-slave-1xrh krb5kdc[17345](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056825, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.195@KRBTEST.COM for kudu/127.28.218.254@KRBTEST.COM
I20260806 22:53:45.381146 17439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:45.372829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.195:43951 (local address 127.28.218.254:36659)
0806 22:53:45.372878 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 22:53:45.372880 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:53:45.373187 (+ 307us) server_negotiation.cc:407] Connection header received
0806 22:53:45.373796 (+ 609us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:53:45.373796 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:53:45.373807 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:53:45.373860 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:53:45.374774 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:45.375344 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:45.376280 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:45.376459 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:45.376513 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:45.378612 (+ 2099us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:53:45.378613 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:53:45.378614 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:53:45.378619 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:53:45.380025 (+ 1406us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:53:45.380381 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:53:45.380382 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:53:45.380382 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:53:45.380431 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:53:45.380641 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:53:45.380641 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:53:45.380641 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:53:45.380738 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:53:45.380788 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 22:53:45.381021 (+ 233us) server_negotiation.cc:299] Negotiation successful
0806 22:53:45.381081 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 22:53:45.381316 17876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:45.372828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:36659 (local address 127.28.218.195:43951)
0806 22:53:45.373035 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0806 22:53:45.373086 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0806 22:53:45.373683 (+ 597us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:53:45.373914 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:53:45.373917 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:53:45.374493 (+ 576us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:53:45.374683 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:53:45.374687 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:53:45.375419 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:53:45.375419 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:53:45.376207 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:53:45.376209 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:53:45.376530 (+ 321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:53:45.376530 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:53:45.376626 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:45.377153 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:53:45.377163 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:53:45.378535 (+ 1372us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:53:45.380097 (+ 1562us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:53:45.380103 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:53:45.380104 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:53:45.380314 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:53:45.380490 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:53:45.380491 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:53:45.380491 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:53:45.380584 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:53:45.380791 (+ 207us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:53:45.380795 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:53:45.380903 (+ 108us) client_negotiation.cc:769] Sending connection context
0806 22:53:45.380992 (+ 89us) client_negotiation.cc:240] Negotiation successful
0806 22:53:45.381108 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":106,"threads_started":1}
I20260806 22:53:45.381783 17874 heartbeater.cc:347] Connected to a master server at 127.28.218.254:36659
I20260806 22:53:45.381886 17874 heartbeater.cc:464] Registering TS with master...
I20260806 22:53:45.382061 17874 heartbeater.cc:511] Master 127.28.218.254:36659 requested a full tablet report, sending...
I20260806 22:53:45.382431 17392 ts_manager.cc:194] Registered new tserver with Master: 7ecaf1d8dddd4344af263a49db100370 (127.28.218.195:37837)
I20260806 22:53:45.382794 17392 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.218.195@KRBTEST.COM'} at 127.28.218.195:43951
I20260806 22:53:45.384810 29547 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:53:45.392275 17439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:45.386677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38402 (local address 127.28.218.254:36659)
0806 22:53:45.386731 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0806 22:53:45.386733 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:53:45.386925 (+ 192us) server_negotiation.cc:407] Connection header received
0806 22:53:45.386946 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:53:45.386946 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:53:45.386962 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:53:45.386996 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:53:45.387598 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:45.388054 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:45.388739 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:45.388913 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:45.388976 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:45.390097 (+ 1121us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:53:45.390097 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:53:45.390098 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:53:45.390102 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:53:45.391292 (+ 1190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:53:45.391596 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:53:45.391597 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:53:45.391597 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:53:45.391645 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:53:45.391840 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:53:45.391840 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:53:45.391841 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:53:45.391972 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:53:45.392046 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 22:53:45.392135 (+ 89us) server_negotiation.cc:299] Negotiation successful
0806 22:53:45.392189 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 22:53:45.393694 17392 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38402:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:53:45.395187 17392 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:53:45.401484 17887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:45.397845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60930 (local address 127.28.218.195:37837)
0806 22:53:45.398021 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0806 22:53:45.398025 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:53:45.398036 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:53:45.398074 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:53:45.398076 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:53:45.398090 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:53:45.398338 (+ 248us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:53:45.398394 (+ 56us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:45.399516 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:45.400584 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:45.401141 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:45.401221 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:45.401270 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 22:53:45.401361 (+ 91us) server_negotiation.cc:299] Negotiation successful
0806 22:53:45.401397 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":116,"threads_started":1}
I20260806 22:53:45.401515 17883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:45.397447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.195:37837 (local address 127.0.0.1:60930)
0806 22:53:45.397686 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0806 22:53:45.397698 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:53:45.397750 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:53:45.398202 (+ 452us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:53:45.398204 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:53:45.398205 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:53:45.398258 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:53:45.398259 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:53:45.399622 (+ 1363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:53:45.399623 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:53:45.400500 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:53:45.400502 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:53:45.401225 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:53:45.401226 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:53:45.401273 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:45.401275 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 22:53:45.401333 (+ 58us) client_negotiation.cc:240] Negotiation successful
0806 22:53:45.401414 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":224,"mutex_wait_us":5,"thread_start_us":81,"threads_started":1}
I20260806 22:53:45.401734 17885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:45.397486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39542 (local address 127.28.218.193:33573)
0806 22:53:45.397710 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0806 22:53:45.397714 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:53:45.397725 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:53:45.397789 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:53:45.397791 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:53:45.397806 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:53:45.397881 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:53:45.398060 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:45.398920 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:45.400253 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:45.400948 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:45.401054 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:45.401115 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 22:53:45.401613 (+ 498us) server_negotiation.cc:299] Negotiation successful
0806 22:53:45.401656 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":186,"thread_start_us":109,"threads_started":1}
I20260806 22:53:45.402187 17886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:45.397616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.194:45091 (local address 127.0.0.1:40552)
0806 22:53:45.398258 (+ 642us) negotiation.cc:107] Waiting for socket to connect
0806 22:53:45.398269 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:53:45.398312 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:53:45.398595 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:53:45.398596 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:53:45.398598 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:53:45.398635 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:53:45.398636 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:53:45.399847 (+ 1211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:53:45.399848 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:53:45.400921 (+ 1073us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:53:45.400924 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:53:45.401958 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:53:45.401959 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:53:45.402004 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:45.402007 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:53:45.402083 (+ 76us) client_negotiation.cc:240] Negotiation successful
0806 22:53:45.402122 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":631,"thread_start_us":40,"threads_started":1}
I20260806 22:53:45.402223 17889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:45.398085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40552 (local address 127.28.218.194:45091)
0806 22:53:45.398424 (+ 339us) server_negotiation.cc:206] Beginning negotiation
0806 22:53:45.398428 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:53:45.398437 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:53:45.398478 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:53:45.398480 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:53:45.398494 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:53:45.398576 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:53:45.398719 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:45.399745 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:45.401070 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:45.401797 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:45.401871 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:45.401921 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 22:53:45.402099 (+ 178us) server_negotiation.cc:299] Negotiation successful
0806 22:53:45.402141 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305,"thread_start_us":96,"threads_started":1}
I20260806 22:53:45.402233 17802 tablet_service.cc:1467] Processing CreateTablet for tablet 26ac6a9b944b4bb797952a849fb2e4a1 (DEFAULT_TABLE table=test-table [id=30e9320965384a00b4e083491f351d62]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:53:45.402635 17802 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26ac6a9b944b4bb797952a849fb2e4a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:53:45.402796 17661 tablet_service.cc:1467] Processing CreateTablet for tablet 26ac6a9b944b4bb797952a849fb2e4a1 (DEFAULT_TABLE table=test-table [id=30e9320965384a00b4e083491f351d62]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:53:45.403160 17661 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26ac6a9b944b4bb797952a849fb2e4a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:53:45.401515 17884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:45.397341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.193:33573 (local address 127.0.0.1:39542)
0806 22:53:45.397586 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0806 22:53:45.397628 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 22:53:45.397696 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:53:45.397891 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:53:45.397894 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:53:45.397897 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:53:45.397974 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:53:45.397975 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:53:45.399020 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:53:45.399021 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:53:45.400162 (+ 1141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:53:45.400165 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:53:45.401153 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:53:45.401153 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:53:45.401244 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:45.401259 (+ 15us) client_negotiation.cc:769] Sending connection context
0806 22:53:45.401333 (+ 74us) client_negotiation.cc:240] Negotiation successful
0806 22:53:45.401414 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":216,"mutex_wait_us":36,"thread_start_us":79,"threads_started":1}
I20260806 22:53:45.403987 17520 tablet_service.cc:1467] Processing CreateTablet for tablet 26ac6a9b944b4bb797952a849fb2e4a1 (DEFAULT_TABLE table=test-table [id=30e9320965384a00b4e083491f351d62]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:53:45.404300 17520 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26ac6a9b944b4bb797952a849fb2e4a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:53:45.406525 17891 tablet_bootstrap.cc:492] T 26ac6a9b944b4bb797952a849fb2e4a1 P 9396e51d73b94aefb87f493ab3dc061b: Bootstrap starting.
I20260806 22:53:45.406594 17890 tablet_bootstrap.cc:492] T 26ac6a9b944b4bb797952a849fb2e4a1 P 7ecaf1d8dddd4344af263a49db100370: Bootstrap starting.
I20260806 22:53:45.407068 17891 tablet_bootstrap.cc:654] T 26ac6a9b944b4bb797952a849fb2e4a1 P 9396e51d73b94aefb87f493ab3dc061b: Neither blocks nor log segments found. Creating new log.
I20260806 22:53:45.407248 17890 tablet_bootstrap.cc:654] T 26ac6a9b944b4bb797952a849fb2e4a1 P 7ecaf1d8dddd4344af263a49db100370: Neither blocks nor log segments found. Creating new log.
I20260806 22:53:45.407662 17891 log.cc:824] T 26ac6a9b944b4bb797952a849fb2e4a1 P 9396e51d73b94aefb87f493ab3dc061b: Log is configured to *not* fsync() on all Append() calls
I20260806 22:53:45.407845 17890 log.cc:824] T 26ac6a9b944b4bb797952a849fb2e4a1 P 7ecaf1d8dddd4344af263a49db100370: Log is configured to *not* fsync() on all Append() calls
I20260806 22:53:45.407903 17892 tablet_bootstrap.cc:492] T 26ac6a9b944b4bb797952a849fb2e4a1 P fac9a6c2a58940cca49b33029ea8d892: Bootstrap starting.
I20260806 22:53:45.408641 17890 tablet_bootstrap.cc:492] T 26ac6a9b944b4bb797952a849fb2e4a1 P 7ecaf1d8dddd4344af263a49db100370: No bootstrap required, opened a new log
I20260806 22:53:45.408641 17892 tablet_bootstrap.cc:654] T 26ac6a9b944b4bb797952a849fb2e4a1 P fac9a6c2a58940cca49b33029ea8d892: Neither blocks nor log segments found. Creating new log.
I20260806 22:53:45.408715 17890 ts_tablet_manager.cc:1397] T 26ac6a9b944b4bb797952a849fb2e4a1 P 7ecaf1d8dddd4344af263a49db100370: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:53:45.409219 17892 log.cc:824] T 26ac6a9b944b4bb797952a849fb2e4a1 P fac9a6c2a58940cca49b33029ea8d892: Log is configured to *not* fsync() on all Append() calls
I20260806 22:53:45.409945 17892 tablet_bootstrap.cc:492] T 26ac6a9b944b4bb797952a849fb2e4a1 P fac9a6c2a58940cca49b33029ea8d892: No bootstrap required, opened a new log
I20260806 22:53:45.410030 17892 ts_tablet_manager.cc:1397] T 26ac6a9b944b4bb797952a849fb2e4a1 P fac9a6c2a58940cca49b33029ea8d892: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:53:45.410754 17890 raft_consensus.cc:348] T 26ac6a9b944b4bb797952a849fb2e4a1 P 7ecaf1d8dddd4344af263a49db100370 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fac9a6c2a58940cca49b33029ea8d892" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 33573 } } peers { permanent_uuid: "9396e51d73b94aefb87f493ab3dc061b" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 45091 } } peers { permanent_uuid: "7ecaf1d8dddd4344af263a49db100370" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 37837 } }
I20260806 22:53:45.410938 17890 raft_consensus.cc:374] T 26ac6a9b944b4bb797952a849fb2e4a1 P 7ecaf1d8dddd4344af263a49db100370 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:53:45.410975 17890 raft_consensus.cc:729] T 26ac6a9b944b4bb797952a849fb2e4a1 P 7ecaf1d8dddd4344af263a49db100370 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ecaf1d8dddd4344af263a49db100370, State: Initialized, Role: FOLLOWER
I20260806 22:53:45.411092 17890 consensus_queue.cc:260] T 26ac6a9b944b4bb797952a849fb2e4a1 P 7ecaf1d8dddd4344af263a49db100370 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fac9a6c2a58940cca49b33029ea8d892" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 33573 } } peers { permanent_uuid: "9396e51d73b94aefb87f493ab3dc061b" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 45091 } } peers { permanent_uuid: "7ecaf1d8dddd4344af263a49db100370" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 37837 } }
I20260806 22:53:45.411404 17890 ts_tablet_manager.cc:1428] T 26ac6a9b944b4bb797952a849fb2e4a1 P 7ecaf1d8dddd4344af263a49db100370: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 22:53:45.411424 17874 heartbeater.cc:503] Master 127.28.218.254:36659 was elected leader, sending a full tablet report...
I20260806 22:53:45.411514 17891 tablet_bootstrap.cc:492] T 26ac6a9b944b4bb797952a849fb2e4a1 P 9396e51d73b94aefb87f493ab3dc061b: No bootstrap required, opened a new log
I20260806 22:53:45.411583 17891 ts_tablet_manager.cc:1397] T 26ac6a9b944b4bb797952a849fb2e4a1 P 9396e51d73b94aefb87f493ab3dc061b: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260806 22:53:45.412142 17892 raft_consensus.cc:348] T 26ac6a9b944b4bb797952a849fb2e4a1 P fac9a6c2a58940cca49b33029ea8d892 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fac9a6c2a58940cca49b33029ea8d892" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 33573 } } peers { permanent_uuid: "9396e51d73b94aefb87f493ab3dc061b" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 45091 } } peers { permanent_uuid: "7ecaf1d8dddd4344af263a49db100370" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 37837 } }
I20260806 22:53:45.412308 17892 raft_consensus.cc:374] T 26ac6a9b944b4bb797952a849fb2e4a1 P fac9a6c2a58940cca49b33029ea8d892 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:53:45.412358 17892 raft_consensus.cc:729] T 26ac6a9b944b4bb797952a849fb2e4a1 P fac9a6c2a58940cca49b33029ea8d892 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fac9a6c2a58940cca49b33029ea8d892, State: Initialized, Role: FOLLOWER
I20260806 22:53:45.412479 17892 consensus_queue.cc:260] T 26ac6a9b944b4bb797952a849fb2e4a1 P fac9a6c2a58940cca49b33029ea8d892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fac9a6c2a58940cca49b33029ea8d892" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 33573 } } peers { permanent_uuid: "9396e51d73b94aefb87f493ab3dc061b" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 45091 } } peers { permanent_uuid: "7ecaf1d8dddd4344af263a49db100370" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 37837 } }
I20260806 22:53:45.412782 17589 heartbeater.cc:503] Master 127.28.218.254:36659 was elected leader, sending a full tablet report...
I20260806 22:53:45.412770 17892 ts_tablet_manager.cc:1428] T 26ac6a9b944b4bb797952a849fb2e4a1 P fac9a6c2a58940cca49b33029ea8d892: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 22:53:45.413594 17891 raft_consensus.cc:348] T 26ac6a9b944b4bb797952a849fb2e4a1 P 9396e51d73b94aefb87f493ab3dc061b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fac9a6c2a58940cca49b33029ea8d892" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 33573 } } peers { permanent_uuid: "9396e51d73b94aefb87f493ab3dc061b" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 45091 } } peers { permanent_uuid: "7ecaf1d8dddd4344af263a49db100370" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 37837 } }
I20260806 22:53:45.413774 17891 raft_consensus.cc:374] T 26ac6a9b944b4bb797952a849fb2e4a1 P 9396e51d73b94aefb87f493ab3dc061b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:53:45.413807 17891 raft_consensus.cc:729] T 26ac6a9b944b4bb797952a849fb2e4a1 P 9396e51d73b94aefb87f493ab3dc061b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9396e51d73b94aefb87f493ab3dc061b, State: Initialized, Role: FOLLOWER
I20260806 22:53:45.413883 17891 consensus_queue.cc:260] T 26ac6a9b944b4bb797952a849fb2e4a1 P 9396e51d73b94aefb87f493ab3dc061b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fac9a6c2a58940cca49b33029ea8d892" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 33573 } } peers { permanent_uuid: "9396e51d73b94aefb87f493ab3dc061b" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 45091 } } peers { permanent_uuid: "7ecaf1d8dddd4344af263a49db100370" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 37837 } }
I20260806 22:53:45.414072 17891 ts_tablet_manager.cc:1428] T 26ac6a9b944b4bb797952a849fb2e4a1 P 9396e51d73b94aefb87f493ab3dc061b: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 22:53:45.414177 17731 heartbeater.cc:503] Master 127.28.218.254:36659 was elected leader, sending a full tablet report...
W20260806 22:53:45.455641 17732 tablet.cc:2369] T 26ac6a9b944b4bb797952a849fb2e4a1 P 9396e51d73b94aefb87f493ab3dc061b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 22:53:45.556206 17590 tablet.cc:2369] T 26ac6a9b944b4bb797952a849fb2e4a1 P fac9a6c2a58940cca49b33029ea8d892: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 22:53:45.622644 17875 tablet.cc:2369] T 26ac6a9b944b4bb797952a849fb2e4a1 P 7ecaf1d8dddd4344af263a49db100370: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:53:45.797670 17897 raft_consensus.cc:482] T 26ac6a9b944b4bb797952a849fb2e4a1 P fac9a6c2a58940cca49b33029ea8d892 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:53:45.797811 17897 raft_consensus.cc:504] T 26ac6a9b944b4bb797952a849fb2e4a1 P fac9a6c2a58940cca49b33029ea8d892 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fac9a6c2a58940cca49b33029ea8d892" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 33573 } } peers { permanent_uuid: "9396e51d73b94aefb87f493ab3dc061b" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 45091 } } peers { permanent_uuid: "7ecaf1d8dddd4344af263a49db100370" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 37837 } }
I20260806 22:53:45.798133 17897 leader_election.cc:288] T 26ac6a9b944b4bb797952a849fb2e4a1 P fac9a6c2a58940cca49b33029ea8d892 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9396e51d73b94aefb87f493ab3dc061b (127.28.218.194:45091), 7ecaf1d8dddd4344af263a49db100370 (127.28.218.195:37837)
I20260806 22:53:45.802587 17889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:45.798482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.193:48529 (local address 127.28.218.194:45091)
0806 22:53:45.798579 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0806 22:53:45.798582 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:53:45.798685 (+ 103us) server_negotiation.cc:407] Connection header received
0806 22:53:45.798734 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:53:45.798735 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:53:45.798749 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:53:45.798826 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:53:45.799058 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:45.799931 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:45.801592 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:45.802180 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:45.802267 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:45.802277 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 22:53:45.802474 (+ 197us) server_negotiation.cc:299] Negotiation successful
0806 22:53:45.802512 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 22:53:45.802587 17887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:45.798533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.193:37907 (local address 127.28.218.195:37837)
0806 22:53:45.798593 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0806 22:53:45.798595 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:53:45.798724 (+ 129us) server_negotiation.cc:407] Connection header received
0806 22:53:45.798754 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:53:45.798756 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:53:45.798766 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:53:45.798822 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:53:45.799152 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:45.799928 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:45.801592 (+ 1664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:45.802189 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:45.802267 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:45.802277 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 22:53:45.802474 (+ 197us) server_negotiation.cc:299] Negotiation successful
0806 22:53:45.802523 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260806 22:53:45.802606 17899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:45.798400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.195:37837 (local address 127.28.218.193:37907)
0806 22:53:45.798592 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0806 22:53:45.798601 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:53:45.798675 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:53:45.798849 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:53:45.798850 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:53:45.798854 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:53:45.799018 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:53:45.799020 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:53:45.800022 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:53:45.800023 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:53:45.801459 (+ 1436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:53:45.801462 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:53:45.802289 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:53:45.802290 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:53:45.802377 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:45.802384 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 22:53:45.802444 (+ 60us) client_negotiation.cc:240] Negotiation successful
0806 22:53:45.802498 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"mutex_wait_us":134,"thread_start_us":79,"threads_started":1}
I20260806 22:53:45.802609 17900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:45.798398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.194:45091 (local address 127.28.218.193:48529)
0806 22:53:45.798572 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0806 22:53:45.798587 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 22:53:45.798675 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:53:45.798849 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:53:45.798850 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:53:45.798852 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:53:45.798963 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:53:45.798966 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:53:45.800022 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:53:45.800023 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:53:45.801459 (+ 1436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:53:45.801462 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:53:45.802288 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:53:45.802289 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:53:45.802377 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:45.802385 (+ 8us) client_negotiation.cc:769] Sending connection context
0806 22:53:45.802444 (+ 59us) client_negotiation.cc:240] Negotiation successful
0806 22:53:45.802498 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"mutex_wait_us":95,"thread_start_us":95,"threads_started":1}
I20260806 22:53:45.803025 17823 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26ac6a9b944b4bb797952a849fb2e4a1" candidate_uuid: "fac9a6c2a58940cca49b33029ea8d892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ecaf1d8dddd4344af263a49db100370" is_pre_election: true
I20260806 22:53:45.803049 17681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26ac6a9b944b4bb797952a849fb2e4a1" candidate_uuid: "fac9a6c2a58940cca49b33029ea8d892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9396e51d73b94aefb87f493ab3dc061b" is_pre_election: true
I20260806 22:53:45.803210 17823 raft_consensus.cc:2454] T 26ac6a9b944b4bb797952a849fb2e4a1 P 7ecaf1d8dddd4344af263a49db100370 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fac9a6c2a58940cca49b33029ea8d892 in term 0.
I20260806 22:53:45.803265 17681 raft_consensus.cc:2454] T 26ac6a9b944b4bb797952a849fb2e4a1 P 9396e51d73b94aefb87f493ab3dc061b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fac9a6c2a58940cca49b33029ea8d892 in term 0.
I20260806 22:53:45.803480 17474 leader_election.cc:302] T 26ac6a9b944b4bb797952a849fb2e4a1 P fac9a6c2a58940cca49b33029ea8d892 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7ecaf1d8dddd4344af263a49db100370, fac9a6c2a58940cca49b33029ea8d892; no voters:
I20260806 22:53:45.803635 17897 raft_consensus.cc:2781] T 26ac6a9b944b4bb797952a849fb2e4a1 P fac9a6c2a58940cca49b33029ea8d892 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:53:45.803695 17897 raft_consensus.cc:482] T 26ac6a9b944b4bb797952a849fb2e4a1 P fac9a6c2a58940cca49b33029ea8d892 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:53:45.803747 17897 raft_consensus.cc:3037] T 26ac6a9b944b4bb797952a849fb2e4a1 P fac9a6c2a58940cca49b33029ea8d892 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:53:45.804484 17897 raft_consensus.cc:504] T 26ac6a9b944b4bb797952a849fb2e4a1 P fac9a6c2a58940cca49b33029ea8d892 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fac9a6c2a58940cca49b33029ea8d892" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 33573 } } peers { permanent_uuid: "9396e51d73b94aefb87f493ab3dc061b" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 45091 } } peers { permanent_uuid: "7ecaf1d8dddd4344af263a49db100370" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 37837 } }
I20260806 22:53:45.804630 17897 leader_election.cc:288] T 26ac6a9b944b4bb797952a849fb2e4a1 P fac9a6c2a58940cca49b33029ea8d892 [CANDIDATE]: Term 1 election: Requested vote from peers 9396e51d73b94aefb87f493ab3dc061b (127.28.218.194:45091), 7ecaf1d8dddd4344af263a49db100370 (127.28.218.195:37837)
I20260806 22:53:45.804814 17823 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26ac6a9b944b4bb797952a849fb2e4a1" candidate_uuid: "fac9a6c2a58940cca49b33029ea8d892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ecaf1d8dddd4344af263a49db100370"
I20260806 22:53:45.804827 17681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26ac6a9b944b4bb797952a849fb2e4a1" candidate_uuid: "fac9a6c2a58940cca49b33029ea8d892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9396e51d73b94aefb87f493ab3dc061b"
I20260806 22:53:45.804899 17681 raft_consensus.cc:3037] T 26ac6a9b944b4bb797952a849fb2e4a1 P 9396e51d73b94aefb87f493ab3dc061b [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:53:45.804899 17823 raft_consensus.cc:3037] T 26ac6a9b944b4bb797952a849fb2e4a1 P 7ecaf1d8dddd4344af263a49db100370 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:53:45.805787 17681 raft_consensus.cc:2454] T 26ac6a9b944b4bb797952a849fb2e4a1 P 9396e51d73b94aefb87f493ab3dc061b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fac9a6c2a58940cca49b33029ea8d892 in term 1.
I20260806 22:53:45.805787 17823 raft_consensus.cc:2454] T 26ac6a9b944b4bb797952a849fb2e4a1 P 7ecaf1d8dddd4344af263a49db100370 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fac9a6c2a58940cca49b33029ea8d892 in term 1.
I20260806 22:53:45.805958 17474 leader_election.cc:302] T 26ac6a9b944b4bb797952a849fb2e4a1 P fac9a6c2a58940cca49b33029ea8d892 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7ecaf1d8dddd4344af263a49db100370, fac9a6c2a58940cca49b33029ea8d892; no voters:
I20260806 22:53:45.806098 17897 raft_consensus.cc:2781] T 26ac6a9b944b4bb797952a849fb2e4a1 P fac9a6c2a58940cca49b33029ea8d892 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:53:45.806238 17897 raft_consensus.cc:686] T 26ac6a9b944b4bb797952a849fb2e4a1 P fac9a6c2a58940cca49b33029ea8d892 [term 1 LEADER]: Becoming Leader. State: Replica: fac9a6c2a58940cca49b33029ea8d892, State: Running, Role: LEADER
I20260806 22:53:45.806342 17897 consensus_queue.cc:237] T 26ac6a9b944b4bb797952a849fb2e4a1 P fac9a6c2a58940cca49b33029ea8d892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fac9a6c2a58940cca49b33029ea8d892" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 33573 } } peers { permanent_uuid: "9396e51d73b94aefb87f493ab3dc061b" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 45091 } } peers { permanent_uuid: "7ecaf1d8dddd4344af263a49db100370" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 37837 } }
I20260806 22:53:45.807305 17392 catalog_manager.cc:5495] T 26ac6a9b944b4bb797952a849fb2e4a1 P fac9a6c2a58940cca49b33029ea8d892 reported cstate change: term changed from 0 to 1, leader changed from <none> to fac9a6c2a58940cca49b33029ea8d892 (127.28.218.193). New cstate: current_term: 1 leader_uuid: "fac9a6c2a58940cca49b33029ea8d892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fac9a6c2a58940cca49b33029ea8d892" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 33573 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9396e51d73b94aefb87f493ab3dc061b" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 45091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7ecaf1d8dddd4344af263a49db100370" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 37837 } health_report { overall_health: UNKNOWN } } }
I20260806 22:53:45.840219 17885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:45.837743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39556 (local address 127.28.218.193:33573)
0806 22:53:45.837802 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0806 22:53:45.837804 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:53:45.837811 (+ 7us) server_negotiation.cc:407] Connection header received
0806 22:53:45.837847 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:53:45.837848 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:53:45.837869 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:53:45.837946 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:53:45.838073 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:45.838543 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:45.839427 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:45.839638 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:45.839717 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:45.839889 (+ 172us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:53:45.840000 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:53:45.840099 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 22:53:45.840118 (+ 19us) server_negotiation.cc:299] Negotiation successful
0806 22:53:45.840136 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 22:53:45.841909 17823 raft_consensus.cc:1261] T 26ac6a9b944b4bb797952a849fb2e4a1 P 7ecaf1d8dddd4344af263a49db100370 [term 1 FOLLOWER]: Refusing update from remote peer fac9a6c2a58940cca49b33029ea8d892: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:53:45.841909 17681 raft_consensus.cc:1261] T 26ac6a9b944b4bb797952a849fb2e4a1 P 9396e51d73b94aefb87f493ab3dc061b [term 1 FOLLOWER]: Refusing update from remote peer fac9a6c2a58940cca49b33029ea8d892: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:53:45.842303 17897 consensus_queue.cc:1035] T 26ac6a9b944b4bb797952a849fb2e4a1 P fac9a6c2a58940cca49b33029ea8d892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ecaf1d8dddd4344af263a49db100370" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 37837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:53:45.842489 17901 consensus_queue.cc:1035] T 26ac6a9b944b4bb797952a849fb2e4a1 P fac9a6c2a58940cca49b33029ea8d892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9396e51d73b94aefb87f493ab3dc061b" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 45091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:53:45.845683 17906 mvcc.cc:205] Tried to move back new op lower bound from 7315688758645809152 to 7315688758503530496. Current Snapshot: MvccSnapshot[applied={T|T < 7315688758645809152}]
I20260806 22:53:45.847712 17392 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38402:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:53:45.847841 17392 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38402:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:53:45.848767 17392 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bd39ee413b9944a896931cdbeb937d08: Sending DeleteTablet for 3 replicas of tablet 26ac6a9b944b4bb797952a849fb2e4a1
I20260806 22:53:45.849046 17520 tablet_service.cc:1514] Processing DeleteTablet for tablet 26ac6a9b944b4bb797952a849fb2e4a1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:53:45 UTC) from {username='kudu', principal='kudu/127.28.218.254@KRBTEST.COM'} at 127.0.0.1:39542
I20260806 22:53:45.849294 17661 tablet_service.cc:1514] Processing DeleteTablet for tablet 26ac6a9b944b4bb797952a849fb2e4a1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:53:45 UTC) from {username='kudu', principal='kudu/127.28.218.254@KRBTEST.COM'} at 127.0.0.1:40552
I20260806 22:53:45.849627 17802 tablet_service.cc:1514] Processing DeleteTablet for tablet 26ac6a9b944b4bb797952a849fb2e4a1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:53:45 UTC) from {username='kudu', principal='kudu/127.28.218.254@KRBTEST.COM'} at 127.0.0.1:60930
I20260806 22:53:45.849869 17916 tablet_replica.cc:330] T 26ac6a9b944b4bb797952a849fb2e4a1 P 7ecaf1d8dddd4344af263a49db100370: stopping tablet replica
I20260806 22:53:45.849982 17916 raft_consensus.cc:2229] T 26ac6a9b944b4bb797952a849fb2e4a1 P 7ecaf1d8dddd4344af263a49db100370 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:53:45.850080 17916 raft_consensus.cc:2258] T 26ac6a9b944b4bb797952a849fb2e4a1 P 7ecaf1d8dddd4344af263a49db100370 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:53:45.850585 17916 ts_tablet_manager.cc:1905] T 26ac6a9b944b4bb797952a849fb2e4a1 P 7ecaf1d8dddd4344af263a49db100370: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:53:45.850754 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 17455
I20260806 22:53:45.853084 17915 tablet_replica.cc:330] T 26ac6a9b944b4bb797952a849fb2e4a1 P 9396e51d73b94aefb87f493ab3dc061b: stopping tablet replica
I20260806 22:53:45.853253 17915 raft_consensus.cc:2229] T 26ac6a9b944b4bb797952a849fb2e4a1 P 9396e51d73b94aefb87f493ab3dc061b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:53:45.853420 17915 raft_consensus.cc:2258] T 26ac6a9b944b4bb797952a849fb2e4a1 P 9396e51d73b94aefb87f493ab3dc061b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:53:45.853595 17916 ts_tablet_manager.cc:1918] T 26ac6a9b944b4bb797952a849fb2e4a1 P 7ecaf1d8dddd4344af263a49db100370: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:53:45.853664 17916 log.cc:1196] T 26ac6a9b944b4bb797952a849fb2e4a1 P 7ecaf1d8dddd4344af263a49db100370: Deleting WAL directory at /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056586788339-29547-0/minicluster-data/ts-2/wal/wals/26ac6a9b944b4bb797952a849fb2e4a1
I20260806 22:53:45.853942 17916 ts_tablet_manager.cc:1939] T 26ac6a9b944b4bb797952a849fb2e4a1 P 7ecaf1d8dddd4344af263a49db100370: Deleting consensus metadata
I20260806 22:53:45.854378 17377 catalog_manager.cc:4841] TS 7ecaf1d8dddd4344af263a49db100370 (127.28.218.195:37837): tablet 26ac6a9b944b4bb797952a849fb2e4a1 (table test-table [id=30e9320965384a00b4e083491f351d62]) successfully deleted
I20260806 22:53:45.855418 17915 ts_tablet_manager.cc:1905] T 26ac6a9b944b4bb797952a849fb2e4a1 P 9396e51d73b94aefb87f493ab3dc061b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:53:45.857312 17915 ts_tablet_manager.cc:1918] T 26ac6a9b944b4bb797952a849fb2e4a1 P 9396e51d73b94aefb87f493ab3dc061b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:53:45.857388 17915 log.cc:1196] T 26ac6a9b944b4bb797952a849fb2e4a1 P 9396e51d73b94aefb87f493ab3dc061b: Deleting WAL directory at /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056586788339-29547-0/minicluster-data/ts-1/wal/wals/26ac6a9b944b4bb797952a849fb2e4a1
I20260806 22:53:45.857631 17915 ts_tablet_manager.cc:1939] T 26ac6a9b944b4bb797952a849fb2e4a1 P 9396e51d73b94aefb87f493ab3dc061b: Deleting consensus metadata
I20260806 22:53:45.858031 17378 catalog_manager.cc:4841] TS 9396e51d73b94aefb87f493ab3dc061b (127.28.218.194:45091): tablet 26ac6a9b944b4bb797952a849fb2e4a1 (table test-table [id=30e9320965384a00b4e083491f351d62]) successfully deleted
W20260806 22:53:45.859827 17378 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.28.218.193:33573 (error 108)
I20260806 22:53:45.860073 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 17594
I20260806 22:53:45.864243 17884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:45.864082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.193:33573 (local address 127.0.0.1:39572)
0806 22:53:45.864148 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0806 22:53:45.864199 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.193:33573: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260806 22:53:45.864425 17378 catalog_manager.cc:4584] TS fac9a6c2a58940cca49b33029ea8d892 (127.28.218.193:33573): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 26ac6a9b944b4bb797952a849fb2e4a1: Network error: Client connection negotiation failed: client connection to 127.28.218.193:33573: connect: Connection refused (error 111)
I20260806 22:53:45.867345 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 17736
I20260806 22:53:45.874238 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 17358
2026-08-06T22:53:45Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2290 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-06T22:53:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:53:45Z Disabled control of system clock
I20260806 22:53:45.909035 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056586788339-29547-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056586788339-29547-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056586788339-29547-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.218.254:41799
--webserver_interface=127.28.218.254
--webserver_port=0
--builtin_ntp_servers=127.28.218.212:35831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.218.254:41799
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 22:53:45.988919 17921 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:53:45.989118 17921 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:53:45.989138 17921 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:53:45.990259 17921 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:53:45.990301 17921 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:53:45.990314 17921 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:53:45.990326 17921 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:53:45.990382 17921 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:53:45.991735 17921 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:35831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056586788339-29547-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056586788339-29547-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.218.254:41799
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.218.254:41799
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.218.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17921
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056586788339-29547-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:53:45.991984 17921 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:53:45.992295 17921 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 22:53:45.997115 17921 server_base.cc:1034] running on GCE node
W20260806 22:53:45.997225 17927 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:53:45.997121 17928 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:53:45.997121 17930 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:53:45.997576 17921 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:53:45.997900 17921 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:53:45.999065 17921 hybrid_clock.cc:648] HybridClock initialized: now 1786056825999058 us; error 45 us; skew 500 ppm
I20260806 22:53:46.000265 17921 webserver.cc:460] Webserver started at http://127.28.218.254:42709/ using document root <none> and password file <none>
I20260806 22:53:46.000531 17921 fs_manager.cc:359] Metadata directory not provided
I20260806 22:53:46.000607 17921 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:53:46.000763 17921 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:53:46.001925 17921 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056586788339-29547-0/minicluster-data/master-0/data/instance:
uuid: "6935bcca503f4bf79f299b276e01ec07"
format_stamp: "Formatted at 2026-08-06 22:53:45 on dist-test-slave-1xrh"
server_key: "d3e1edadcee30fc2fd2b7a4acda5f17f"
server_key_iv: "4eed42dae55868f980eb9d55fa01ffe6"
server_key_version: "encryptionkey@0"
I20260806 22:53:46.002301 17921 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056586788339-29547-0/minicluster-data/master-0/wal/instance:
uuid: "6935bcca503f4bf79f299b276e01ec07"
format_stamp: "Formatted at 2026-08-06 22:53:45 on dist-test-slave-1xrh"
server_key: "d3e1edadcee30fc2fd2b7a4acda5f17f"
server_key_iv: "4eed42dae55868f980eb9d55fa01ffe6"
server_key_version: "encryptionkey@0"
I20260806 22:53:46.003669 17921 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:53:46.004437 17936 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:53:46.004621 17921 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:53:46.004726 17921 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056586788339-29547-0/minicluster-data/master-0/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056586788339-29547-0/minicluster-data/master-0/wal
uuid: "6935bcca503f4bf79f299b276e01ec07"
format_stamp: "Formatted at 2026-08-06 22:53:45 on dist-test-slave-1xrh"
server_key: "d3e1edadcee30fc2fd2b7a4acda5f17f"
server_key_iv: "4eed42dae55868f980eb9d55fa01ffe6"
server_key_version: "encryptionkey@0"
I20260806 22:53:46.004809 17921 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056586788339-29547-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056586788339-29547-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056586788339-29547-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:53:46.017710 17921 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:53:46.018015 17921 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:53:46.018167 17921 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:53:46.022110 17921 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.254:41799
I20260806 22:53:46.022141 17992 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.254:41799 every 8 connection(s)
I20260806 22:53:46.022475 17921 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
I20260806 22:53:46.023367 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 17921
I20260806 22:53:46.023535 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056586788339-29547-0/minicluster-data/master-0/wal/instance
I20260806 22:53:46.023630 17993 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:53:46.023772 29547 external_mini_cluster.cc:1428] Setting key f9cbc787e4c925e8d7015060e78fdb55
I20260806 22:53:46.026674 17993 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6935bcca503f4bf79f299b276e01ec07: Bootstrap starting.
I20260806 22:53:46.027392 17993 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6935bcca503f4bf79f299b276e01ec07: Neither blocks nor log segments found. Creating new log.
I20260806 22:53:46.027793 17993 log.cc:824] T 00000000000000000000000000000000 P 6935bcca503f4bf79f299b276e01ec07: Log is configured to *not* fsync() on all Append() calls
I20260806 22:53:46.028513 17993 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6935bcca503f4bf79f299b276e01ec07: No bootstrap required, opened a new log
I20260806 22:53:46.029546 17996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:46.024336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37534 (local address 127.28.218.254:41799)
0806 22:53:46.024534 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0806 22:53:46.024541 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:53:46.024566 (+ 25us) server_negotiation.cc:407] Connection header received
0806 22:53:46.025109 (+ 543us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:53:46.025115 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:53:46.025172 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:53:46.025281 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:53:46.025507 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:46.026340 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:46.027465 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:46.027928 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:46.028048 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:46.028807 (+ 759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:53:46.028811 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:53:46.028811 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:53:46.029009 (+ 198us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:53:46.029056 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:53:46.029081 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:53:46.029148 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 22:53:46.029240 (+ 92us) server_negotiation.cc:299] Negotiation successful
0806 22:53:46.029317 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":82,"threads_started":1}
I20260806 22:53:46.030794 17993 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6935bcca503f4bf79f299b276e01ec07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6935bcca503f4bf79f299b276e01ec07" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 41799 } }
I20260806 22:53:46.030921 17993 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6935bcca503f4bf79f299b276e01ec07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:53:46.030942 17993 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6935bcca503f4bf79f299b276e01ec07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6935bcca503f4bf79f299b276e01ec07, State: Initialized, Role: FOLLOWER
I20260806 22:53:46.031100 17993 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6935bcca503f4bf79f299b276e01ec07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6935bcca503f4bf79f299b276e01ec07" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 41799 } }
I20260806 22:53:46.031231 17993 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6935bcca503f4bf79f299b276e01ec07 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:53:46.031287 17993 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6935bcca503f4bf79f299b276e01ec07 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:53:46.031337 17993 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6935bcca503f4bf79f299b276e01ec07 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:53:46.031862 17993 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6935bcca503f4bf79f299b276e01ec07 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6935bcca503f4bf79f299b276e01ec07" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 41799 } }
I20260806 22:53:46.031976 17993 leader_election.cc:302] T 00000000000000000000000000000000 P 6935bcca503f4bf79f299b276e01ec07 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6935bcca503f4bf79f299b276e01ec07; no voters:
I20260806 22:53:46.032194 17993 leader_election.cc:288] T 00000000000000000000000000000000 P 6935bcca503f4bf79f299b276e01ec07 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:53:46.032253 17998 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6935bcca503f4bf79f299b276e01ec07 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:53:46.032431 17998 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6935bcca503f4bf79f299b276e01ec07 [term 1 LEADER]: Becoming Leader. State: Replica: 6935bcca503f4bf79f299b276e01ec07, State: Running, Role: LEADER
I20260806 22:53:46.032492 17993 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6935bcca503f4bf79f299b276e01ec07 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:53:46.032524 17998 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6935bcca503f4bf79f299b276e01ec07 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6935bcca503f4bf79f299b276e01ec07" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 41799 } }
I20260806 22:53:46.032964 18000 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6935bcca503f4bf79f299b276e01ec07 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6935bcca503f4bf79f299b276e01ec07. Latest consensus state: current_term: 1 leader_uuid: "6935bcca503f4bf79f299b276e01ec07" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6935bcca503f4bf79f299b276e01ec07" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 41799 } } }
I20260806 22:53:46.033073 18000 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6935bcca503f4bf79f299b276e01ec07 [sys.catalog]: This master's current role is: LEADER
I20260806 22:53:46.032944 17999 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6935bcca503f4bf79f299b276e01ec07 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6935bcca503f4bf79f299b276e01ec07" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6935bcca503f4bf79f299b276e01ec07" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 41799 } } }
I20260806 22:53:46.033289 17999 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6935bcca503f4bf79f299b276e01ec07 [sys.catalog]: This master's current role is: LEADER
I20260806 22:53:46.033548 18006 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:53:46.034066 18006 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:53:46.035604 18006 catalog_manager.cc:1347] Generated new cluster ID: 4e501e87b6564f44a3f9e68ec7099998
I20260806 22:53:46.035656 18006 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:53:46.048455 18006 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:53:46.048964 18006 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:53:46.054070 18006 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6935bcca503f4bf79f299b276e01ec07: Generated new TSK 0
I20260806 22:53:46.054255 18006 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:53:46.058089 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056586788339-29547-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056586788339-29547-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056586788339-29547-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056586788339-29547-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.218.193:0
--local_ip_for_outbound_sockets=127.28.218.193
--webserver_interface=127.28.218.193
--webserver_port=0
--tserver_master_addrs=127.28.218.254:41799
--builtin_ntp_servers=127.28.218.212:35831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 22:53:46.137717 18017 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:53:46.137903 18017 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:53:46.137924 18017 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:53:46.139048 18017 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:53:46.139087 18017 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:53:46.139111 18017 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.218.193
I20260806 22:53:46.140435 18017 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:35831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056586788339-29547-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056586788339-29547-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.218.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056586788339-29547-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.218.193
--webserver_port=0
--tserver_master_addrs=127.28.218.254:41799
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18017
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.218.193
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056586788339-29547-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:53:46.140681 18017 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:53:46.140998 18017 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:53:46.146009 18025 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:53:46.146078 18023 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:53:46.146020 18022 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:53:46.146476 18017 server_base.cc:1034] running on GCE node
I20260806 22:53:46.146674 18017 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:53:46.146903 18017 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:53:46.148056 18017 hybrid_clock.cc:648] HybridClock initialized: now 1786056826148039 us; error 29 us; skew 500 ppm
I20260806 22:53:46.149354 18017 webserver.cc:460] Webserver started at http://127.28.218.193:43527/ using document root <none> and password file <none>
I20260806 22:53:46.149583 18017 fs_manager.cc:359] Metadata directory not provided
I20260806 22:53:46.149633 18017 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:53:46.149749 18017 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:53:46.151038 18017 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056586788339-29547-0/minicluster-data/ts-0/data/instance:
uuid: "b730c0aa6fcd4f5988da3e43fcfadcbe"
format_stamp: "Formatted at 2026-08-06 22:53:46 on dist-test-slave-1xrh"
server_key: "c95d2010fe64c783454d5583b4cdc805"
server_key_iv: "4327b9b6792bab18efb6ac16c4f44b30"
server_key_version: "encryptionkey@0"
I20260806 22:53:46.151492 18017 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056586788339-29547-0/minicluster-data/ts-0/wal/instance:
uuid: "b730c0aa6fcd4f5988da3e43fcfadcbe"
format_stamp: "Formatted at 2026-08-06 22:53:46 on dist-test-slave-1xrh"
server_key: "c95d2010fe64c783454d5583b4cdc805"
server_key_iv: "4327b9b6792bab18efb6ac16c4f44b30"
server_key_version: "encryptionkey@0"
I20260806 22:53:46.153105 18017 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:53:46.153908 18031 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:53:46.154202 18017 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:53:46.154297 18017 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056586788339-29547-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056586788339-29547-0/minicluster-data/ts-0/wal
uuid: "b730c0aa6fcd4f5988da3e43fcfadcbe"
format_stamp: "Formatted at 2026-08-06 22:53:46 on dist-test-slave-1xrh"
server_key: "c95d2010fe64c783454d5583b4cdc805"
server_key_iv: "4327b9b6792bab18efb6ac16c4f44b30"
server_key_version: "encryptionkey@0"
I20260806 22:53:46.154402 18017 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056586788339-29547-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056586788339-29547-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056586788339-29547-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:53:46.177667 18017 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:53:46.177922 18017 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:53:46.178022 18017 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:53:46.178196 18017 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:53:46.178555 18017 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:53:46.178591 18017 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:53:46.178650 18017 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:53:46.178675 18017 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:53:46.184646 18017 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.193:46127
I20260806 22:53:46.184701 18150 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.193:46127 every 8 connection(s)
I20260806 22:53:46.185015 18017 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056586788339-29547-0/minicluster-data/ts-0/data/info.pb
I20260806 22:53:46.189755 17996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:46.185740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.193:54655 (local address 127.28.218.254:41799)
0806 22:53:46.185829 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0806 22:53:46.185831 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:53:46.186054 (+ 223us) server_negotiation.cc:407] Connection header received
0806 22:53:46.186511 (+ 457us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:53:46.186512 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:53:46.186527 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:53:46.186581 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:53:46.187215 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:46.187691 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:46.188573 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:46.188753 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:46.188801 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:46.189533 (+ 732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:53:46.189534 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:53:46.189534 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:53:46.189538 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:53:46.189551 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:53:46.189554 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:53:46.189596 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0806 22:53:46.189675 (+ 79us) server_negotiation.cc:299] Negotiation successful
0806 22:53:46.189697 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 22:53:46.189961 18153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:46.185712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:41799 (local address 127.28.218.193:54655)
0806 22:53:46.185924 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0806 22:53:46.185958 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0806 22:53:46.186427 (+ 469us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:53:46.186646 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:53:46.186649 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:53:46.186657 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:53:46.187146 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:53:46.187150 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:53:46.187769 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:53:46.187769 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:53:46.188501 (+ 732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:53:46.188503 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:53:46.188825 (+ 322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:53:46.188826 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:53:46.188925 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:46.189400 (+ 475us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:53:46.189400 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:53:46.189427 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:53:46.189447 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:53:46.189607 (+ 160us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:53:46.189611 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:53:46.189613 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 22:53:46.189662 (+ 49us) client_negotiation.cc:240] Negotiation successful
0806 22:53:46.189733 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":114,"threads_started":1}
I20260806 22:53:46.190481 18151 heartbeater.cc:347] Connected to a master server at 127.28.218.254:41799
I20260806 22:53:46.190583 18151 heartbeater.cc:464] Registering TS with master...
I20260806 22:53:46.190752 18151 heartbeater.cc:511] Master 127.28.218.254:41799 requested a full tablet report, sending...
I20260806 22:53:46.191274 17954 ts_manager.cc:194] Registered new tserver with Master: b730c0aa6fcd4f5988da3e43fcfadcbe (127.28.218.193:46127)
I20260806 22:53:46.191905 17954 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.28.218.193:54655
I20260806 22:53:46.193394 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 18017
I20260806 22:53:46.193513 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056586788339-29547-0/minicluster-data/ts-0/wal/instance
I20260806 22:53:46.193717 29547 external_mini_cluster.cc:1428] Setting key e3770a3ad44eeda96f677fa99ee7e22f
I20260806 22:53:46.194532 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056586788339-29547-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056586788339-29547-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056586788339-29547-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056586788339-29547-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.218.194:0
--local_ip_for_outbound_sockets=127.28.218.194
--webserver_interface=127.28.218.194
--webserver_port=0
--tserver_master_addrs=127.28.218.254:41799
--builtin_ntp_servers=127.28.218.212:35831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 22:53:46.274178 18154 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:53:46.274369 18154 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:53:46.274390 18154 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:53:46.275543 18154 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:53:46.275583 18154 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:53:46.275609 18154 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.218.194
I20260806 22:53:46.276865 18154 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:35831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056586788339-29547-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056586788339-29547-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.218.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056586788339-29547-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.218.194
--webserver_port=0
--tserver_master_addrs=127.28.218.254:41799
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18154
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.218.194
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056586788339-29547-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:53:46.277055 18154 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:53:46.277275 18154 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:53:46.282371 18159 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:53:46.282371 18162 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:53:46.282445 18160 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:53:46.282651 18154 server_base.cc:1034] running on GCE node
I20260806 22:53:46.282864 18154 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:53:46.283092 18154 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:53:46.284286 18154 hybrid_clock.cc:648] HybridClock initialized: now 1786056826284279 us; error 37 us; skew 500 ppm
I20260806 22:53:46.285504 18154 webserver.cc:460] Webserver started at http://127.28.218.194:39915/ using document root <none> and password file <none>
I20260806 22:53:46.285737 18154 fs_manager.cc:359] Metadata directory not provided
I20260806 22:53:46.285806 18154 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:53:46.285935 18154 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:53:46.286913 18154 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056586788339-29547-0/minicluster-data/ts-1/data/instance:
uuid: "9c34e23d0e2b4076a3e136c8fbb76b31"
format_stamp: "Formatted at 2026-08-06 22:53:46 on dist-test-slave-1xrh"
server_key: "8fbd1079dd4d10d9b562df8d2a4dd759"
server_key_iv: "04e99fb61e48ca9c0419b42f8cee37b4"
server_key_version: "encryptionkey@0"
I20260806 22:53:46.287315 18154 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056586788339-29547-0/minicluster-data/ts-1/wal/instance:
uuid: "9c34e23d0e2b4076a3e136c8fbb76b31"
format_stamp: "Formatted at 2026-08-06 22:53:46 on dist-test-slave-1xrh"
server_key: "8fbd1079dd4d10d9b562df8d2a4dd759"
server_key_iv: "04e99fb61e48ca9c0419b42f8cee37b4"
server_key_version: "encryptionkey@0"
I20260806 22:53:46.288570 18154 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:53:46.289261 18169 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:53:46.289433 18154 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:53:46.289512 18154 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056586788339-29547-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056586788339-29547-0/minicluster-data/ts-1/wal
uuid: "9c34e23d0e2b4076a3e136c8fbb76b31"
format_stamp: "Formatted at 2026-08-06 22:53:46 on dist-test-slave-1xrh"
server_key: "8fbd1079dd4d10d9b562df8d2a4dd759"
server_key_iv: "04e99fb61e48ca9c0419b42f8cee37b4"
server_key_version: "encryptionkey@0"
I20260806 22:53:46.289577 18154 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056586788339-29547-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056586788339-29547-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056586788339-29547-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:53:46.315048 18154 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:53:46.315409 18154 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:53:46.315567 18154 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:53:46.315796 18154 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:53:46.316130 18154 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:53:46.316164 18154 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:53:46.316184 18154 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:53:46.316197 18154 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:53:46.321866 18154 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.194:39787
I20260806 22:53:46.321905 18291 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.194:39787 every 8 connection(s)
I20260806 22:53:46.322237 18154 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056586788339-29547-0/minicluster-data/ts-1/data/info.pb
I20260806 22:53:46.327199 17996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:46.323018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.194:45377 (local address 127.28.218.254:41799)
0806 22:53:46.323101 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0806 22:53:46.323104 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:53:46.323371 (+ 267us) server_negotiation.cc:407] Connection header received
0806 22:53:46.323829 (+ 458us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:53:46.323830 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:53:46.323839 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:53:46.323920 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:53:46.324555 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:46.325050 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:46.325940 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:46.326116 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:46.326164 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:46.326973 (+ 809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:53:46.326974 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:53:46.326974 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:53:46.326979 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:53:46.326994 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:53:46.326998 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:53:46.327039 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0806 22:53:46.327112 (+ 73us) server_negotiation.cc:299] Negotiation successful
0806 22:53:46.327142 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 22:53:46.327456 18294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:46.323015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:41799 (local address 127.28.218.194:45377)
0806 22:53:46.323250 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0806 22:53:46.323283 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0806 22:53:46.323736 (+ 453us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:53:46.323981 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:53:46.323984 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:53:46.323993 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:53:46.324474 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:53:46.324478 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:53:46.325127 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:53:46.325128 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:53:46.325865 (+ 737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:53:46.325866 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:53:46.326189 (+ 323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:53:46.326190 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:53:46.326292 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:46.326846 (+ 554us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:53:46.326846 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:53:46.326872 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:53:46.326892 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:53:46.327056 (+ 164us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:53:46.327060 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:53:46.327062 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 22:53:46.327108 (+ 46us) client_negotiation.cc:240] Negotiation successful
0806 22:53:46.327233 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":122,"threads_started":1}
I20260806 22:53:46.328050 18292 heartbeater.cc:347] Connected to a master server at 127.28.218.254:41799
I20260806 22:53:46.328150 18292 heartbeater.cc:464] Registering TS with master...
I20260806 22:53:46.328321 18292 heartbeater.cc:511] Master 127.28.218.254:41799 requested a full tablet report, sending...
I20260806 22:53:46.328697 17954 ts_manager.cc:194] Registered new tserver with Master: 9c34e23d0e2b4076a3e136c8fbb76b31 (127.28.218.194:39787)
I20260806 22:53:46.329072 17954 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.28.218.194:45377
I20260806 22:53:46.329984 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 18154
I20260806 22:53:46.330109 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056586788339-29547-0/minicluster-data/ts-1/wal/instance
I20260806 22:53:46.330308 29547 external_mini_cluster.cc:1428] Setting key a5973a53f7673af39f48f5a70067fd73
I20260806 22:53:46.331308 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056586788339-29547-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056586788339-29547-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056586788339-29547-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056586788339-29547-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.218.195:0
--local_ip_for_outbound_sockets=127.28.218.195
--webserver_interface=127.28.218.195
--webserver_port=0
--tserver_master_addrs=127.28.218.254:41799
--builtin_ntp_servers=127.28.218.212:35831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 22:53:46.413247 18295 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:53:46.413488 18295 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:53:46.413511 18295 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:53:46.414705 18295 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:53:46.414743 18295 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:53:46.414767 18295 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.218.195
I20260806 22:53:46.416101 18295 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:35831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056586788339-29547-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056586788339-29547-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.218.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056586788339-29547-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.218.195
--webserver_port=0
--tserver_master_addrs=127.28.218.254:41799
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18295
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.218.195
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056586788339-29547-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:53:46.416334 18295 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:53:46.416604 18295 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:53:46.421453 18300 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:53:46.421476 18295 server_base.cc:1034] running on GCE node
W20260806 22:53:46.421449 18301 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:53:46.421451 18303 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:53:46.421939 18295 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:53:46.422190 18295 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:53:46.423329 18295 hybrid_clock.cc:648] HybridClock initialized: now 1786056826423306 us; error 41 us; skew 500 ppm
I20260806 22:53:46.424538 18295 webserver.cc:460] Webserver started at http://127.28.218.195:41043/ using document root <none> and password file <none>
I20260806 22:53:46.424798 18295 fs_manager.cc:359] Metadata directory not provided
I20260806 22:53:46.424844 18295 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:53:46.425030 18295 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:53:46.426079 18295 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056586788339-29547-0/minicluster-data/ts-2/data/instance:
uuid: "320bba54f87d4da2b34cce01e4e4f25f"
format_stamp: "Formatted at 2026-08-06 22:53:46 on dist-test-slave-1xrh"
server_key: "311e8994ef76ae55d9e794bf97bd4d93"
server_key_iv: "4294d7d89c5a9fffc714df74e651ab3f"
server_key_version: "encryptionkey@0"
I20260806 22:53:46.426501 18295 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056586788339-29547-0/minicluster-data/ts-2/wal/instance:
uuid: "320bba54f87d4da2b34cce01e4e4f25f"
format_stamp: "Formatted at 2026-08-06 22:53:46 on dist-test-slave-1xrh"
server_key: "311e8994ef76ae55d9e794bf97bd4d93"
server_key_iv: "4294d7d89c5a9fffc714df74e651ab3f"
server_key_version: "encryptionkey@0"
I20260806 22:53:46.427840 18295 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:53:46.428521 18309 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:53:46.428690 18295 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:53:46.428807 18295 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056586788339-29547-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056586788339-29547-0/minicluster-data/ts-2/wal
uuid: "320bba54f87d4da2b34cce01e4e4f25f"
format_stamp: "Formatted at 2026-08-06 22:53:46 on dist-test-slave-1xrh"
server_key: "311e8994ef76ae55d9e794bf97bd4d93"
server_key_iv: "4294d7d89c5a9fffc714df74e651ab3f"
server_key_version: "encryptionkey@0"
I20260806 22:53:46.428884 18295 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056586788339-29547-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056586788339-29547-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056586788339-29547-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:53:46.453229 18295 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:53:46.453554 18295 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:53:46.453729 18295 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:53:46.454005 18295 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:53:46.454350 18295 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:53:46.454409 18295 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:53:46.454452 18295 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:53:46.454481 18295 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:53:46.460289 18295 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.195:35967
I20260806 22:53:46.460366 18429 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.195:35967 every 8 connection(s)
I20260806 22:53:46.460697 18295 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056586788339-29547-0/minicluster-data/ts-2/data/info.pb
I20260806 22:53:46.465651 17996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:46.461491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.195:55467 (local address 127.28.218.254:41799)
0806 22:53:46.461566 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0806 22:53:46.461568 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:53:46.461793 (+ 225us) server_negotiation.cc:407] Connection header received
0806 22:53:46.462263 (+ 470us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:53:46.462264 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:53:46.462274 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:53:46.462327 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:53:46.462990 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:46.463539 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:46.464479 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:46.464655 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:46.464704 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:46.465432 (+ 728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:53:46.465433 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:53:46.465433 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:53:46.465437 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:53:46.465451 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:53:46.465454 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:53:46.465504 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 22:53:46.465562 (+ 58us) server_negotiation.cc:299] Negotiation successful
0806 22:53:46.465583 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 22:53:46.465860 18432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:46.461474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:41799 (local address 127.28.218.195:55467)
0806 22:53:46.461670 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0806 22:53:46.461703 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0806 22:53:46.462178 (+ 475us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:53:46.462375 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:53:46.462379 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:53:46.462386 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:53:46.462872 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:53:46.462876 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:53:46.463615 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:53:46.463616 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:53:46.464396 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:53:46.464398 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:53:46.464729 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:53:46.464730 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:53:46.464832 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:46.465306 (+ 474us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:53:46.465306 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:53:46.465332 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:53:46.465353 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:53:46.465504 (+ 151us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:53:46.465506 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:53:46.465508 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 22:53:46.465553 (+ 45us) client_negotiation.cc:240] Negotiation successful
0806 22:53:46.465624 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":94,"threads_started":1}
I20260806 22:53:46.466425 18430 heartbeater.cc:347] Connected to a master server at 127.28.218.254:41799
I20260806 22:53:46.466526 18430 heartbeater.cc:464] Registering TS with master...
I20260806 22:53:46.466701 18430 heartbeater.cc:511] Master 127.28.218.254:41799 requested a full tablet report, sending...
I20260806 22:53:46.466820 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 18295
I20260806 22:53:46.466962 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056586788339-29547-0/minicluster-data/ts-2/wal/instance
I20260806 22:53:46.467111 17954 ts_manager.cc:194] Registered new tserver with Master: 320bba54f87d4da2b34cce01e4e4f25f (127.28.218.195:35967)
I20260806 22:53:46.467216 29547 external_mini_cluster.cc:1428] Setting key 1b34a3bec55c847ff3cdbe95bd9767b9
I20260806 22:53:46.467535 17954 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.28.218.195:55467
I20260806 22:53:46.468215 29547 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:53:46.472116 17996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:46.469511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37536 (local address 127.28.218.254:41799)
0806 22:53:46.469553 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 22:53:46.469555 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:53:46.469685 (+ 130us) server_negotiation.cc:407] Connection header received
0806 22:53:46.469716 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:53:46.469717 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:53:46.469724 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:53:46.469767 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:53:46.469885 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:46.470365 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:46.471080 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:46.471322 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:46.471373 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:46.471909 (+ 536us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:53:46.471910 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:53:46.471910 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:53:46.471913 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:53:46.471925 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:53:46.471928 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:53:46.471968 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0806 22:53:46.472002 (+ 34us) server_negotiation.cc:299] Negotiation successful
0806 22:53:46.472019 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 22:53:46.473429 17954 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37536:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:53:46.474977 17954 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:53:46.480947 18440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:46.477130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.194:39787 (local address 127.0.0.1:60454)
0806 22:53:46.477288 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0806 22:53:46.477298 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:53:46.477337 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:53:46.478039 (+ 702us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:53:46.478040 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:53:46.478042 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:53:46.478087 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:53:46.478092 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:53:46.479076 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:53:46.479077 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:53:46.479997 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:53:46.480000 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:53:46.480662 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:53:46.480663 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:53:46.480736 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:46.480749 (+ 13us) client_negotiation.cc:769] Sending connection context
0806 22:53:46.480822 (+ 73us) client_negotiation.cc:240] Negotiation successful
0806 22:53:46.480876 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"mutex_wait_us":78,"thread_start_us":96,"threads_started":1}
I20260806 22:53:46.480959 18441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:46.477369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60454 (local address 127.28.218.194:39787)
0806 22:53:46.477801 (+ 432us) server_negotiation.cc:206] Beginning negotiation
0806 22:53:46.477805 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:53:46.477816 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:53:46.477871 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:53:46.477873 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:53:46.477889 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:53:46.477965 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:53:46.478171 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:46.479005 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:46.480097 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:46.480569 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:46.480662 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:46.480749 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 22:53:46.480836 (+ 87us) server_negotiation.cc:299] Negotiation successful
0806 22:53:46.480876 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":388,"thread_start_us":93,"threads_started":1}
I20260806 22:53:46.481581 18439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:46.477134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.193:46127 (local address 127.0.0.1:35328)
0806 22:53:46.477436 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0806 22:53:46.477444 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 22:53:46.477480 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:53:46.477973 (+ 493us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:53:46.477974 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:53:46.477977 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:53:46.478027 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:53:46.478029 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:53:46.479258 (+ 1229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:53:46.479259 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:53:46.480330 (+ 1071us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:53:46.480332 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:53:46.481330 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:53:46.481331 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:53:46.481395 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:46.481400 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:53:46.481447 (+ 47us) client_negotiation.cc:240] Negotiation successful
0806 22:53:46.481500 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":90,"threads_started":1}
I20260806 22:53:46.481686 18438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:46.477123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.195:35967 (local address 127.0.0.1:43180)
0806 22:53:46.477232 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0806 22:53:46.477271 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 22:53:46.477337 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:53:46.477697 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:53:46.477700 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:53:46.477702 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:53:46.477763 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:53:46.477765 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:53:46.479019 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:53:46.479020 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:53:46.480577 (+ 1557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:53:46.480579 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:53:46.481432 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:53:46.481433 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:53:46.481486 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:46.481488 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 22:53:46.481517 (+ 29us) client_negotiation.cc:240] Negotiation successful
0806 22:53:46.481549 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":88,"mutex_wait_us":61,"thread_start_us":67,"threads_started":1}
I20260806 22:53:46.481843 18442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:46.477371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43180 (local address 127.28.218.195:35967)
0806 22:53:46.477535 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0806 22:53:46.477539 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:53:46.477548 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:53:46.477578 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:53:46.477579 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:53:46.477598 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:53:46.477678 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:53:46.477848 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:46.478925 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:46.480664 (+ 1739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:46.481344 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:46.481623 (+ 279us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:46.481714 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 22:53:46.481739 (+ 25us) server_negotiation.cc:299] Negotiation successful
0806 22:53:46.481775 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":93,"threads_started":1}
I20260806 22:53:46.481873 18443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:46.477561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35328 (local address 127.28.218.193:46127)
0806 22:53:46.477723 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0806 22:53:46.477728 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:53:46.477739 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:53:46.477781 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:53:46.477783 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:53:46.477796 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:53:46.477901 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:53:46.478113 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:46.479174 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:46.480416 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:46.481121 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:46.481191 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:46.481272 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 22:53:46.481755 (+ 483us) server_negotiation.cc:299] Negotiation successful
0806 22:53:46.481795 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":98,"threads_started":1}
I20260806 22:53:46.482020 18220 tablet_service.cc:1467] Processing CreateTablet for tablet ded5ff8f91ba4db586fb68eca29e0708 (DEFAULT_TABLE table=test-table [id=e656d3ed716c492eac7420ca357f45ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:53:46.482381 18220 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ded5ff8f91ba4db586fb68eca29e0708. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:53:46.482484 18360 tablet_service.cc:1467] Processing CreateTablet for tablet ded5ff8f91ba4db586fb68eca29e0708 (DEFAULT_TABLE table=test-table [id=e656d3ed716c492eac7420ca357f45ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:53:46.482503 18081 tablet_service.cc:1467] Processing CreateTablet for tablet ded5ff8f91ba4db586fb68eca29e0708 (DEFAULT_TABLE table=test-table [id=e656d3ed716c492eac7420ca357f45ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:53:46.482712 18360 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ded5ff8f91ba4db586fb68eca29e0708. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:53:46.482805 18081 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ded5ff8f91ba4db586fb68eca29e0708. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:53:46.485256 18444 tablet_bootstrap.cc:492] T ded5ff8f91ba4db586fb68eca29e0708 P 320bba54f87d4da2b34cce01e4e4f25f: Bootstrap starting.
I20260806 22:53:46.485790 18445 tablet_bootstrap.cc:492] T ded5ff8f91ba4db586fb68eca29e0708 P 9c34e23d0e2b4076a3e136c8fbb76b31: Bootstrap starting.
I20260806 22:53:46.485848 18444 tablet_bootstrap.cc:654] T ded5ff8f91ba4db586fb68eca29e0708 P 320bba54f87d4da2b34cce01e4e4f25f: Neither blocks nor log segments found. Creating new log.
I20260806 22:53:46.486178 18444 log.cc:824] T ded5ff8f91ba4db586fb68eca29e0708 P 320bba54f87d4da2b34cce01e4e4f25f: Log is configured to *not* fsync() on all Append() calls
I20260806 22:53:46.486399 18445 tablet_bootstrap.cc:654] T ded5ff8f91ba4db586fb68eca29e0708 P 9c34e23d0e2b4076a3e136c8fbb76b31: Neither blocks nor log segments found. Creating new log.
I20260806 22:53:46.486527 18446 tablet_bootstrap.cc:492] T ded5ff8f91ba4db586fb68eca29e0708 P b730c0aa6fcd4f5988da3e43fcfadcbe: Bootstrap starting.
I20260806 22:53:46.486723 18445 log.cc:824] T ded5ff8f91ba4db586fb68eca29e0708 P 9c34e23d0e2b4076a3e136c8fbb76b31: Log is configured to *not* fsync() on all Append() calls
I20260806 22:53:46.486953 18444 tablet_bootstrap.cc:492] T ded5ff8f91ba4db586fb68eca29e0708 P 320bba54f87d4da2b34cce01e4e4f25f: No bootstrap required, opened a new log
I20260806 22:53:46.487038 18444 ts_tablet_manager.cc:1397] T ded5ff8f91ba4db586fb68eca29e0708 P 320bba54f87d4da2b34cce01e4e4f25f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:53:46.487152 18446 tablet_bootstrap.cc:654] T ded5ff8f91ba4db586fb68eca29e0708 P b730c0aa6fcd4f5988da3e43fcfadcbe: Neither blocks nor log segments found. Creating new log.
I20260806 22:53:46.487454 18446 log.cc:824] T ded5ff8f91ba4db586fb68eca29e0708 P b730c0aa6fcd4f5988da3e43fcfadcbe: Log is configured to *not* fsync() on all Append() calls
I20260806 22:53:46.487507 18445 tablet_bootstrap.cc:492] T ded5ff8f91ba4db586fb68eca29e0708 P 9c34e23d0e2b4076a3e136c8fbb76b31: No bootstrap required, opened a new log
I20260806 22:53:46.487584 18445 ts_tablet_manager.cc:1397] T ded5ff8f91ba4db586fb68eca29e0708 P 9c34e23d0e2b4076a3e136c8fbb76b31: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:53:46.488220 18446 tablet_bootstrap.cc:492] T ded5ff8f91ba4db586fb68eca29e0708 P b730c0aa6fcd4f5988da3e43fcfadcbe: No bootstrap required, opened a new log
I20260806 22:53:46.488289 18446 ts_tablet_manager.cc:1397] T ded5ff8f91ba4db586fb68eca29e0708 P b730c0aa6fcd4f5988da3e43fcfadcbe: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260806 22:53:46.489456 18445 raft_consensus.cc:348] T ded5ff8f91ba4db586fb68eca29e0708 P 9c34e23d0e2b4076a3e136c8fbb76b31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c34e23d0e2b4076a3e136c8fbb76b31" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 39787 } } peers { permanent_uuid: "320bba54f87d4da2b34cce01e4e4f25f" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 35967 } } peers { permanent_uuid: "b730c0aa6fcd4f5988da3e43fcfadcbe" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 46127 } }
I20260806 22:53:46.489617 18445 raft_consensus.cc:374] T ded5ff8f91ba4db586fb68eca29e0708 P 9c34e23d0e2b4076a3e136c8fbb76b31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:53:46.489663 18445 raft_consensus.cc:729] T ded5ff8f91ba4db586fb68eca29e0708 P 9c34e23d0e2b4076a3e136c8fbb76b31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c34e23d0e2b4076a3e136c8fbb76b31, State: Initialized, Role: FOLLOWER
I20260806 22:53:46.489791 18445 consensus_queue.cc:260] T ded5ff8f91ba4db586fb68eca29e0708 P 9c34e23d0e2b4076a3e136c8fbb76b31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c34e23d0e2b4076a3e136c8fbb76b31" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 39787 } } peers { permanent_uuid: "320bba54f87d4da2b34cce01e4e4f25f" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 35967 } } peers { permanent_uuid: "b730c0aa6fcd4f5988da3e43fcfadcbe" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 46127 } }
I20260806 22:53:46.489727 18444 raft_consensus.cc:348] T ded5ff8f91ba4db586fb68eca29e0708 P 320bba54f87d4da2b34cce01e4e4f25f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c34e23d0e2b4076a3e136c8fbb76b31" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 39787 } } peers { permanent_uuid: "320bba54f87d4da2b34cce01e4e4f25f" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 35967 } } peers { permanent_uuid: "b730c0aa6fcd4f5988da3e43fcfadcbe" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 46127 } }
I20260806 22:53:46.489874 18444 raft_consensus.cc:374] T ded5ff8f91ba4db586fb68eca29e0708 P 320bba54f87d4da2b34cce01e4e4f25f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:53:46.489917 18444 raft_consensus.cc:729] T ded5ff8f91ba4db586fb68eca29e0708 P 320bba54f87d4da2b34cce01e4e4f25f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 320bba54f87d4da2b34cce01e4e4f25f, State: Initialized, Role: FOLLOWER
I20260806 22:53:46.490054 18445 ts_tablet_manager.cc:1428] T ded5ff8f91ba4db586fb68eca29e0708 P 9c34e23d0e2b4076a3e136c8fbb76b31: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 22:53:46.490059 18444 consensus_queue.cc:260] T ded5ff8f91ba4db586fb68eca29e0708 P 320bba54f87d4da2b34cce01e4e4f25f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c34e23d0e2b4076a3e136c8fbb76b31" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 39787 } } peers { permanent_uuid: "320bba54f87d4da2b34cce01e4e4f25f" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 35967 } } peers { permanent_uuid: "b730c0aa6fcd4f5988da3e43fcfadcbe" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 46127 } }
I20260806 22:53:46.490172 18292 heartbeater.cc:503] Master 127.28.218.254:41799 was elected leader, sending a full tablet report...
I20260806 22:53:46.490326 18444 ts_tablet_manager.cc:1428] T ded5ff8f91ba4db586fb68eca29e0708 P 320bba54f87d4da2b34cce01e4e4f25f: Time spent starting tablet: real 0.003s user 0.000s sys 0.004s
I20260806 22:53:46.490796 18430 heartbeater.cc:503] Master 127.28.218.254:41799 was elected leader, sending a full tablet report...
I20260806 22:53:46.490724 18446 raft_consensus.cc:348] T ded5ff8f91ba4db586fb68eca29e0708 P b730c0aa6fcd4f5988da3e43fcfadcbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c34e23d0e2b4076a3e136c8fbb76b31" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 39787 } } peers { permanent_uuid: "320bba54f87d4da2b34cce01e4e4f25f" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 35967 } } peers { permanent_uuid: "b730c0aa6fcd4f5988da3e43fcfadcbe" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 46127 } }
I20260806 22:53:46.490865 18446 raft_consensus.cc:374] T ded5ff8f91ba4db586fb68eca29e0708 P b730c0aa6fcd4f5988da3e43fcfadcbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:53:46.490911 18446 raft_consensus.cc:729] T ded5ff8f91ba4db586fb68eca29e0708 P b730c0aa6fcd4f5988da3e43fcfadcbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b730c0aa6fcd4f5988da3e43fcfadcbe, State: Initialized, Role: FOLLOWER
I20260806 22:53:46.491019 18446 consensus_queue.cc:260] T ded5ff8f91ba4db586fb68eca29e0708 P b730c0aa6fcd4f5988da3e43fcfadcbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c34e23d0e2b4076a3e136c8fbb76b31" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 39787 } } peers { permanent_uuid: "320bba54f87d4da2b34cce01e4e4f25f" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 35967 } } peers { permanent_uuid: "b730c0aa6fcd4f5988da3e43fcfadcbe" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 46127 } }
I20260806 22:53:46.491285 18446 ts_tablet_manager.cc:1428] T ded5ff8f91ba4db586fb68eca29e0708 P b730c0aa6fcd4f5988da3e43fcfadcbe: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260806 22:53:46.491436 18151 heartbeater.cc:503] Master 127.28.218.254:41799 was elected leader, sending a full tablet report...
I20260806 22:53:46.496577 18452 raft_consensus.cc:482] T ded5ff8f91ba4db586fb68eca29e0708 P b730c0aa6fcd4f5988da3e43fcfadcbe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:53:46.496644 18452 raft_consensus.cc:504] T ded5ff8f91ba4db586fb68eca29e0708 P b730c0aa6fcd4f5988da3e43fcfadcbe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c34e23d0e2b4076a3e136c8fbb76b31" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 39787 } } peers { permanent_uuid: "320bba54f87d4da2b34cce01e4e4f25f" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 35967 } } peers { permanent_uuid: "b730c0aa6fcd4f5988da3e43fcfadcbe" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 46127 } }
I20260806 22:53:46.496968 18452 leader_election.cc:288] T ded5ff8f91ba4db586fb68eca29e0708 P b730c0aa6fcd4f5988da3e43fcfadcbe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9c34e23d0e2b4076a3e136c8fbb76b31 (127.28.218.194:39787), 320bba54f87d4da2b34cce01e4e4f25f (127.28.218.195:35967)
I20260806 22:53:46.499806 18153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:46.497107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.194:39787 (local address 127.28.218.193:37507)
0806 22:53:46.497221 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0806 22:53:46.497233 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:53:46.497286 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:53:46.497423 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:53:46.497424 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:53:46.497425 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:53:46.497489 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:53:46.497491 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:53:46.498225 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:53:46.498226 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:53:46.499008 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:53:46.499010 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:53:46.499563 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:53:46.499564 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:53:46.499637 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:46.499640 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:53:46.499692 (+ 52us) client_negotiation.cc:240] Negotiation successful
0806 22:53:46.499731 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97}
I20260806 22:53:46.499816 18441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:46.497145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.193:37507 (local address 127.28.218.194:39787)
0806 22:53:46.497186 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0806 22:53:46.497189 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:53:46.497291 (+ 102us) server_negotiation.cc:407] Connection header received
0806 22:53:46.497343 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:53:46.497344 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:53:46.497355 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:53:46.497417 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:53:46.497557 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:46.498148 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:46.499094 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:46.499480 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:46.499532 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:46.499537 (+ 5us) server_negotiation.cc:1035] Waiting for connection context
0806 22:53:46.499706 (+ 169us) server_negotiation.cc:299] Negotiation successful
0806 22:53:46.499745 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 22:53:46.499913 18442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:46.497165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.218.193:37045 (local address 127.28.218.195:35967)
0806 22:53:46.497209 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 22:53:46.497211 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:53:46.497393 (+ 182us) server_negotiation.cc:407] Connection header received
0806 22:53:46.497439 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:53:46.497439 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:53:46.497447 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:53:46.497514 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:53:46.497717 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:46.498248 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:46.499189 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:46.499564 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:46.499634 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:46.499641 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0806 22:53:46.499814 (+ 173us) server_negotiation.cc:299] Negotiation successful
0806 22:53:46.499854 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 22:53:46.500133 18242 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ded5ff8f91ba4db586fb68eca29e0708" candidate_uuid: "b730c0aa6fcd4f5988da3e43fcfadcbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c34e23d0e2b4076a3e136c8fbb76b31" is_pre_election: true
I20260806 22:53:46.500278 18242 raft_consensus.cc:2454] T ded5ff8f91ba4db586fb68eca29e0708 P 9c34e23d0e2b4076a3e136c8fbb76b31 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b730c0aa6fcd4f5988da3e43fcfadcbe in term 0.
I20260806 22:53:46.500406 18453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:46.497113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.195:35967 (local address 127.28.218.193:37045)
0806 22:53:46.497309 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0806 22:53:46.497320 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:53:46.497382 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:53:46.497589 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:53:46.497589 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:53:46.497591 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:53:46.497651 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:53:46.497652 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:53:46.498335 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:53:46.498336 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:53:46.499066 (+ 730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:53:46.499068 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:53:46.499647 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:53:46.499648 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:53:46.499733 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:46.499736 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:53:46.499792 (+ 56us) client_negotiation.cc:240] Negotiation successful
0806 22:53:46.499825 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"mutex_wait_us":38,"thread_start_us":91,"threads_started":1}
I20260806 22:53:46.500538 18033 leader_election.cc:302] T ded5ff8f91ba4db586fb68eca29e0708 P b730c0aa6fcd4f5988da3e43fcfadcbe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9c34e23d0e2b4076a3e136c8fbb76b31, b730c0aa6fcd4f5988da3e43fcfadcbe; no voters:
I20260806 22:53:46.500645 18452 raft_consensus.cc:2781] T ded5ff8f91ba4db586fb68eca29e0708 P b730c0aa6fcd4f5988da3e43fcfadcbe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:53:46.500703 18452 raft_consensus.cc:482] T ded5ff8f91ba4db586fb68eca29e0708 P b730c0aa6fcd4f5988da3e43fcfadcbe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:53:46.500726 18452 raft_consensus.cc:3037] T ded5ff8f91ba4db586fb68eca29e0708 P b730c0aa6fcd4f5988da3e43fcfadcbe [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:53:46.500752 18380 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ded5ff8f91ba4db586fb68eca29e0708" candidate_uuid: "b730c0aa6fcd4f5988da3e43fcfadcbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "320bba54f87d4da2b34cce01e4e4f25f" is_pre_election: true
I20260806 22:53:46.500873 18380 raft_consensus.cc:2454] T ded5ff8f91ba4db586fb68eca29e0708 P 320bba54f87d4da2b34cce01e4e4f25f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b730c0aa6fcd4f5988da3e43fcfadcbe in term 0.
I20260806 22:53:46.501447 18452 raft_consensus.cc:504] T ded5ff8f91ba4db586fb68eca29e0708 P b730c0aa6fcd4f5988da3e43fcfadcbe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c34e23d0e2b4076a3e136c8fbb76b31" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 39787 } } peers { permanent_uuid: "320bba54f87d4da2b34cce01e4e4f25f" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 35967 } } peers { permanent_uuid: "b730c0aa6fcd4f5988da3e43fcfadcbe" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 46127 } }
I20260806 22:53:46.501583 18452 leader_election.cc:288] T ded5ff8f91ba4db586fb68eca29e0708 P b730c0aa6fcd4f5988da3e43fcfadcbe [CANDIDATE]: Term 1 election: Requested vote from peers 9c34e23d0e2b4076a3e136c8fbb76b31 (127.28.218.194:39787), 320bba54f87d4da2b34cce01e4e4f25f (127.28.218.195:35967)
I20260806 22:53:46.501722 18380 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ded5ff8f91ba4db586fb68eca29e0708" candidate_uuid: "b730c0aa6fcd4f5988da3e43fcfadcbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "320bba54f87d4da2b34cce01e4e4f25f"
I20260806 22:53:46.501722 18242 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ded5ff8f91ba4db586fb68eca29e0708" candidate_uuid: "b730c0aa6fcd4f5988da3e43fcfadcbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c34e23d0e2b4076a3e136c8fbb76b31"
I20260806 22:53:46.501791 18380 raft_consensus.cc:3037] T ded5ff8f91ba4db586fb68eca29e0708 P 320bba54f87d4da2b34cce01e4e4f25f [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:53:46.501791 18242 raft_consensus.cc:3037] T ded5ff8f91ba4db586fb68eca29e0708 P 9c34e23d0e2b4076a3e136c8fbb76b31 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:53:46.502609 18380 raft_consensus.cc:2454] T ded5ff8f91ba4db586fb68eca29e0708 P 320bba54f87d4da2b34cce01e4e4f25f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b730c0aa6fcd4f5988da3e43fcfadcbe in term 1.
I20260806 22:53:46.502609 18242 raft_consensus.cc:2454] T ded5ff8f91ba4db586fb68eca29e0708 P 9c34e23d0e2b4076a3e136c8fbb76b31 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b730c0aa6fcd4f5988da3e43fcfadcbe in term 1.
I20260806 22:53:46.502780 18033 leader_election.cc:302] T ded5ff8f91ba4db586fb68eca29e0708 P b730c0aa6fcd4f5988da3e43fcfadcbe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9c34e23d0e2b4076a3e136c8fbb76b31, b730c0aa6fcd4f5988da3e43fcfadcbe; no voters:
I20260806 22:53:46.502887 18452 raft_consensus.cc:2781] T ded5ff8f91ba4db586fb68eca29e0708 P b730c0aa6fcd4f5988da3e43fcfadcbe [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:53:46.503018 18452 raft_consensus.cc:686] T ded5ff8f91ba4db586fb68eca29e0708 P b730c0aa6fcd4f5988da3e43fcfadcbe [term 1 LEADER]: Becoming Leader. State: Replica: b730c0aa6fcd4f5988da3e43fcfadcbe, State: Running, Role: LEADER
I20260806 22:53:46.503108 18452 consensus_queue.cc:237] T ded5ff8f91ba4db586fb68eca29e0708 P b730c0aa6fcd4f5988da3e43fcfadcbe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c34e23d0e2b4076a3e136c8fbb76b31" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 39787 } } peers { permanent_uuid: "320bba54f87d4da2b34cce01e4e4f25f" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 35967 } } peers { permanent_uuid: "b730c0aa6fcd4f5988da3e43fcfadcbe" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 46127 } }
I20260806 22:53:46.503935 17953 catalog_manager.cc:5495] T ded5ff8f91ba4db586fb68eca29e0708 P b730c0aa6fcd4f5988da3e43fcfadcbe reported cstate change: term changed from 0 to 1, leader changed from <none> to b730c0aa6fcd4f5988da3e43fcfadcbe (127.28.218.193). New cstate: current_term: 1 leader_uuid: "b730c0aa6fcd4f5988da3e43fcfadcbe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c34e23d0e2b4076a3e136c8fbb76b31" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 39787 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "320bba54f87d4da2b34cce01e4e4f25f" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 35967 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b730c0aa6fcd4f5988da3e43fcfadcbe" member_type: VOTER last_known_addr { host: "127.28.218.193" port: 46127 } health_report { overall_health: HEALTHY } } }
I20260806 22:53:46.516005 18443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:46.513871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35342 (local address 127.28.218.193:46127)
0806 22:53:46.513933 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0806 22:53:46.513935 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:53:46.513942 (+ 7us) server_negotiation.cc:407] Connection header received
0806 22:53:46.513969 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:53:46.513970 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:53:46.513985 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:53:46.514049 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:53:46.514127 (+ 78us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:46.514551 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:46.515344 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:46.515511 (+ 167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:46.515560 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:46.515691 (+ 131us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:53:46.515771 (+ 80us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:53:46.515826 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 22:53:46.515854 (+ 28us) server_negotiation.cc:299] Negotiation successful
0806 22:53:46.515869 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 22:53:46.517910 18242 raft_consensus.cc:1261] T ded5ff8f91ba4db586fb68eca29e0708 P 9c34e23d0e2b4076a3e136c8fbb76b31 [term 1 FOLLOWER]: Refusing update from remote peer b730c0aa6fcd4f5988da3e43fcfadcbe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:53:46.517901 18380 raft_consensus.cc:1261] T ded5ff8f91ba4db586fb68eca29e0708 P 320bba54f87d4da2b34cce01e4e4f25f [term 1 FOLLOWER]: Refusing update from remote peer b730c0aa6fcd4f5988da3e43fcfadcbe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:53:46.518237 18452 consensus_queue.cc:1035] T ded5ff8f91ba4db586fb68eca29e0708 P b730c0aa6fcd4f5988da3e43fcfadcbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c34e23d0e2b4076a3e136c8fbb76b31" member_type: VOTER last_known_addr { host: "127.28.218.194" port: 39787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:53:46.518368 18454 consensus_queue.cc:1035] T ded5ff8f91ba4db586fb68eca29e0708 P b730c0aa6fcd4f5988da3e43fcfadcbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "320bba54f87d4da2b34cce01e4e4f25f" member_type: VOTER last_known_addr { host: "127.28.218.195" port: 35967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:53:46.520807 18459 mvcc.cc:205] Tried to move back new op lower bound from 7315688761414909952 to 7315688761357410304. Current Snapshot: MvccSnapshot[applied={T|T < 7315688761414909952}]
I20260806 22:53:46.523335 17953 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37536:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:53:46.523406 17953 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37536:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:53:46.524195 17953 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6935bcca503f4bf79f299b276e01ec07: Sending DeleteTablet for 3 replicas of tablet ded5ff8f91ba4db586fb68eca29e0708
I20260806 22:53:46.524510 18220 tablet_service.cc:1514] Processing DeleteTablet for tablet ded5ff8f91ba4db586fb68eca29e0708 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:53:46 UTC) from {username='slave'} at 127.0.0.1:60454
I20260806 22:53:46.524529 18360 tablet_service.cc:1514] Processing DeleteTablet for tablet ded5ff8f91ba4db586fb68eca29e0708 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:53:46 UTC) from {username='slave'} at 127.0.0.1:43180
I20260806 22:53:46.524533 18081 tablet_service.cc:1514] Processing DeleteTablet for tablet ded5ff8f91ba4db586fb68eca29e0708 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:53:46 UTC) from {username='slave'} at 127.0.0.1:35328
I20260806 22:53:46.524729 18467 tablet_replica.cc:330] T ded5ff8f91ba4db586fb68eca29e0708 P 9c34e23d0e2b4076a3e136c8fbb76b31: stopping tablet replica
I20260806 22:53:46.524813 18467 raft_consensus.cc:2229] T ded5ff8f91ba4db586fb68eca29e0708 P 9c34e23d0e2b4076a3e136c8fbb76b31 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:53:46.524892 18467 raft_consensus.cc:2258] T ded5ff8f91ba4db586fb68eca29e0708 P 9c34e23d0e2b4076a3e136c8fbb76b31 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:53:46.525046 18469 tablet_replica.cc:330] T ded5ff8f91ba4db586fb68eca29e0708 P b730c0aa6fcd4f5988da3e43fcfadcbe: stopping tablet replica
I20260806 22:53:46.525084 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 18017
I20260806 22:53:46.525168 18468 tablet_replica.cc:330] T ded5ff8f91ba4db586fb68eca29e0708 P 320bba54f87d4da2b34cce01e4e4f25f: stopping tablet replica
I20260806 22:53:46.525938 18468 raft_consensus.cc:2229] T ded5ff8f91ba4db586fb68eca29e0708 P 320bba54f87d4da2b34cce01e4e4f25f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:53:46.526093 18468 raft_consensus.cc:2258] T ded5ff8f91ba4db586fb68eca29e0708 P 320bba54f87d4da2b34cce01e4e4f25f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:53:46.526659 18468 ts_tablet_manager.cc:1905] T ded5ff8f91ba4db586fb68eca29e0708 P 320bba54f87d4da2b34cce01e4e4f25f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:53:46.526654 18467 ts_tablet_manager.cc:1905] T ded5ff8f91ba4db586fb68eca29e0708 P 9c34e23d0e2b4076a3e136c8fbb76b31: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:53:46.528549 18468 ts_tablet_manager.cc:1918] T ded5ff8f91ba4db586fb68eca29e0708 P 320bba54f87d4da2b34cce01e4e4f25f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:53:46.528646 18468 log.cc:1196] T ded5ff8f91ba4db586fb68eca29e0708 P 320bba54f87d4da2b34cce01e4e4f25f: Deleting WAL directory at /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056586788339-29547-0/minicluster-data/ts-2/wal/wals/ded5ff8f91ba4db586fb68eca29e0708
I20260806 22:53:46.528865 18467 ts_tablet_manager.cc:1918] T ded5ff8f91ba4db586fb68eca29e0708 P 9c34e23d0e2b4076a3e136c8fbb76b31: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:53:46.528890 18468 ts_tablet_manager.cc:1939] T ded5ff8f91ba4db586fb68eca29e0708 P 320bba54f87d4da2b34cce01e4e4f25f: Deleting consensus metadata
I20260806 22:53:46.528941 18467 log.cc:1196] T ded5ff8f91ba4db586fb68eca29e0708 P 9c34e23d0e2b4076a3e136c8fbb76b31: Deleting WAL directory at /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056586788339-29547-0/minicluster-data/ts-1/wal/wals/ded5ff8f91ba4db586fb68eca29e0708
I20260806 22:53:46.529196 18467 ts_tablet_manager.cc:1939] T ded5ff8f91ba4db586fb68eca29e0708 P 9c34e23d0e2b4076a3e136c8fbb76b31: Deleting consensus metadata
I20260806 22:53:46.529582 17937 catalog_manager.cc:4841] TS 320bba54f87d4da2b34cce01e4e4f25f (127.28.218.195:35967): tablet ded5ff8f91ba4db586fb68eca29e0708 (table test-table [id=e656d3ed716c492eac7420ca357f45ce]) successfully deleted
I20260806 22:53:46.529930 17938 catalog_manager.cc:4841] TS 9c34e23d0e2b4076a3e136c8fbb76b31 (127.28.218.194:39787): tablet ded5ff8f91ba4db586fb68eca29e0708 (table test-table [id=e656d3ed716c492eac7420ca357f45ce]) successfully deleted
W20260806 22:53:46.533627 17940 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.28.218.193:46127 (error 108)
I20260806 22:53:46.533867 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 18154
I20260806 22:53:46.534848 18438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:46.534030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.193:46127 (local address 127.0.0.1:35350)
0806 22:53:46.534780 (+ 750us) negotiation.cc:107] Waiting for socket to connect
0806 22:53:46.534823 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.193:46127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":731}
W20260806 22:53:46.535230 17940 catalog_manager.cc:4584] TS b730c0aa6fcd4f5988da3e43fcfadcbe (127.28.218.193:46127): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ded5ff8f91ba4db586fb68eca29e0708: Network error: Client connection negotiation failed: client connection to 127.28.218.193:46127: connect: Connection refused (error 111)
I20260806 22:53:46.539525 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 18295
I20260806 22:53:46.545720 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 17921
2026-08-06T22:53:46Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (665 ms)
[----------] 6 tests from EncryptionPolicyTest (9070 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786056586788339-29547-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:53:46 dist-test-slave-1xrh krb5kdc[18473](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:53:46 dist-test-slave-1xrh krb5kdc[18473](info): set up 2 sockets
Aug 06 22:53:46 dist-test-slave-1xrh krb5kdc[18473](info): commencing operation
krb5kdc: starting...
W20260806 22:53:47.619060 29547 mini_kdc.cc:121] Time spent starting KDC: real 1.053s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786056586788339-29547-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786056586788339-29547-0/krb5kdc/test-user.keytab.
Aug 06 22:53:47 dist-test-slave-1xrh krb5kdc[18473](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056827, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:53:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:53:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786056586788339-29547-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786056586788339-29547-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 22:53:47.738327 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786056586788339-29547-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786056586788339-29547-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786056586788339-29547-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40759
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786056586788339-29547-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40759
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:53:47.823700 18487 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:53:47.823940 18487 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:53:47.823964 18487 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:53:47.825119 18487 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:53:47.825162 18487 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:53:47.825188 18487 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:53:47.825227 18487 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:53:47.825259 18487 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:53:47.827014 18487 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786056586788339-29547-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786056586788339-29547-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40759
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786056586788339-29547-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40759
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18487
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786056586788339-29547-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:53:47.827317 18487 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:53:47.827564 18487 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:53:47.832422 18493 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:53:47.832414 18495 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:53:47.832417 18492 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:53:47.832615 18487 server_base.cc:1034] running on GCE node
I20260806 22:53:47.832996 18487 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:53:47.833348 18487 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:53:47.834604 18487 hybrid_clock.cc:648] HybridClock initialized: now 1786056827834581 us; error 39 us; skew 500 ppm
Aug 06 22:53:47 dist-test-slave-1xrh krb5kdc[18473](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056827, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:53:47.841943 18487 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 22:53:47.842423 18487 webserver.cc:460] Webserver started at http://127.0.0.1:43401/ using document root <none> and password file <none>
I20260806 22:53:47.842653 18487 fs_manager.cc:359] Metadata directory not provided
I20260806 22:53:47.842736 18487 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:53:47.842885 18487 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:53:47.844023 18487 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786056586788339-29547-0/minicluster-data/master-0/data/instance:
uuid: "bb5fa3bbbbb5431dab723c61a7dd0f4f"
format_stamp: "Formatted at 2026-08-06 22:53:47 on dist-test-slave-1xrh"
server_key: "715ed0690feac7bb0504d3c43dfa07fb"
server_key_iv: "7fd01f9e03d6010ca83e51b0a026729d"
server_key_version: "encryptionkey@0"
I20260806 22:53:47.844396 18487 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786056586788339-29547-0/minicluster-data/master-0/wal/instance:
uuid: "bb5fa3bbbbb5431dab723c61a7dd0f4f"
format_stamp: "Formatted at 2026-08-06 22:53:47 on dist-test-slave-1xrh"
server_key: "715ed0690feac7bb0504d3c43dfa07fb"
server_key_iv: "7fd01f9e03d6010ca83e51b0a026729d"
server_key_version: "encryptionkey@0"
I20260806 22:53:47.845757 18487 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 22:53:47.846534 18502 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:53:47.846761 18487 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:53:47.846913 18487 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786056586788339-29547-0/minicluster-data/master-0/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786056586788339-29547-0/minicluster-data/master-0/wal
uuid: "bb5fa3bbbbb5431dab723c61a7dd0f4f"
format_stamp: "Formatted at 2026-08-06 22:53:47 on dist-test-slave-1xrh"
server_key: "715ed0690feac7bb0504d3c43dfa07fb"
server_key_iv: "7fd01f9e03d6010ca83e51b0a026729d"
server_key_version: "encryptionkey@0"
I20260806 22:53:47.847007 18487 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786056586788339-29547-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786056586788339-29547-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786056586788339-29547-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:53:47.857427 18487 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:53:47.859426 18487 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:53:47.859584 18487 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:53:47.864444 18487 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40759
I20260806 22:53:47.864486 18556 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40759 every 8 connection(s)
I20260806 22:53:47.864828 18487 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
I20260806 22:53:47.865660 18557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:53:47.867862 18557 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb5fa3bbbbb5431dab723c61a7dd0f4f: Bootstrap starting.
I20260806 22:53:47.868368 18557 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bb5fa3bbbbb5431dab723c61a7dd0f4f: Neither blocks nor log segments found. Creating new log.
I20260806 22:53:47.868619 18557 log.cc:824] T 00000000000000000000000000000000 P bb5fa3bbbbb5431dab723c61a7dd0f4f: Log is configured to *not* fsync() on all Append() calls
I20260806 22:53:47.869248 18557 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb5fa3bbbbb5431dab723c61a7dd0f4f: No bootstrap required, opened a new log
I20260806 22:53:47.870751 18557 raft_consensus.cc:348] T 00000000000000000000000000000000 P bb5fa3bbbbb5431dab723c61a7dd0f4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb5fa3bbbbb5431dab723c61a7dd0f4f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40759 } }
I20260806 22:53:47.870879 18557 raft_consensus.cc:374] T 00000000000000000000000000000000 P bb5fa3bbbbb5431dab723c61a7dd0f4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:53:47.870901 18557 raft_consensus.cc:729] T 00000000000000000000000000000000 P bb5fa3bbbbb5431dab723c61a7dd0f4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb5fa3bbbbb5431dab723c61a7dd0f4f, State: Initialized, Role: FOLLOWER
I20260806 22:53:47.871001 18557 consensus_queue.cc:260] T 00000000000000000000000000000000 P bb5fa3bbbbb5431dab723c61a7dd0f4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb5fa3bbbbb5431dab723c61a7dd0f4f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40759 } }
I20260806 22:53:47.871053 18557 raft_consensus.cc:388] T 00000000000000000000000000000000 P bb5fa3bbbbb5431dab723c61a7dd0f4f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:53:47.871074 18557 raft_consensus.cc:482] T 00000000000000000000000000000000 P bb5fa3bbbbb5431dab723c61a7dd0f4f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:53:47.871174 18557 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bb5fa3bbbbb5431dab723c61a7dd0f4f [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:53:47.871690 18557 raft_consensus.cc:504] T 00000000000000000000000000000000 P bb5fa3bbbbb5431dab723c61a7dd0f4f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb5fa3bbbbb5431dab723c61a7dd0f4f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40759 } }
I20260806 22:53:47.871803 18557 leader_election.cc:302] T 00000000000000000000000000000000 P bb5fa3bbbbb5431dab723c61a7dd0f4f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bb5fa3bbbbb5431dab723c61a7dd0f4f; no voters:
I20260806 22:53:47.871982 18557 leader_election.cc:288] T 00000000000000000000000000000000 P bb5fa3bbbbb5431dab723c61a7dd0f4f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:53:47.872042 18560 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bb5fa3bbbbb5431dab723c61a7dd0f4f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:53:47.872195 18560 raft_consensus.cc:686] T 00000000000000000000000000000000 P bb5fa3bbbbb5431dab723c61a7dd0f4f [term 1 LEADER]: Becoming Leader. State: Replica: bb5fa3bbbbb5431dab723c61a7dd0f4f, State: Running, Role: LEADER
I20260806 22:53:47.872225 18557 sys_catalog.cc:564] T 00000000000000000000000000000000 P bb5fa3bbbbb5431dab723c61a7dd0f4f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:53:47.872280 18560 consensus_queue.cc:237] T 00000000000000000000000000000000 P bb5fa3bbbbb5431dab723c61a7dd0f4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb5fa3bbbbb5431dab723c61a7dd0f4f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40759 } }
I20260806 22:53:47.872670 18563 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb5fa3bbbbb5431dab723c61a7dd0f4f [sys.catalog]: SysCatalogTable state changed. Reason: New leader bb5fa3bbbbb5431dab723c61a7dd0f4f. Latest consensus state: current_term: 1 leader_uuid: "bb5fa3bbbbb5431dab723c61a7dd0f4f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb5fa3bbbbb5431dab723c61a7dd0f4f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40759 } } }
I20260806 22:53:47.872762 18563 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb5fa3bbbbb5431dab723c61a7dd0f4f [sys.catalog]: This master's current role is: LEADER
I20260806 22:53:47.872794 18561 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb5fa3bbbbb5431dab723c61a7dd0f4f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bb5fa3bbbbb5431dab723c61a7dd0f4f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb5fa3bbbbb5431dab723c61a7dd0f4f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40759 } } }
I20260806 22:53:47.872856 18561 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb5fa3bbbbb5431dab723c61a7dd0f4f [sys.catalog]: This master's current role is: LEADER
I20260806 22:53:47.872987 18569 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:53:47.873118 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 18487
I20260806 22:53:47.873240 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786056586788339-29547-0/minicluster-data/master-0/wal/instance
I20260806 22:53:47.873459 29547 external_mini_cluster.cc:1428] Setting key 5b74fa4325c0ed912f2ef9ee17d02dd1
I20260806 22:53:47.873541 18569 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:53:47.875573 18569 catalog_manager.cc:1347] Generated new cluster ID: c745f7370471491391b02e8c327b857e
I20260806 22:53:47.875629 18569 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 22:53:47 dist-test-slave-1xrh krb5kdc[18473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056827, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 22:53:47.882519 18569 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:53:47.883234 18569 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:53:47.884786 18580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:47.874092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41727 (local address 127.0.0.1:40759)
0806 22:53:47.874252 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0806 22:53:47.874261 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:53:47.874298 (+ 37us) server_negotiation.cc:407] Connection header received
0806 22:53:47.874384 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:53:47.874389 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:53:47.874430 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:53:47.874548 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:53:47.875384 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:47.876647 (+ 1263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:47.877430 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:47.877864 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:47.878094 (+ 230us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:47.880649 (+ 2555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:53:47.880653 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:53:47.880653 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:53:47.880665 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:53:47.883204 (+ 2539us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:53:47.883634 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:53:47.883637 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:53:47.883638 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:53:47.883728 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:53:47.883978 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:53:47.883979 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:53:47.883980 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:53:47.884193 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:53:47.884393 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0806 22:53:47.884478 (+ 85us) server_negotiation.cc:299] Negotiation successful
0806 22:53:47.884580 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":115,"thread_start_us":76,"threads_started":1}
I20260806 22:53:47.885792 29547 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:53:47.889655 18569 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bb5fa3bbbbb5431dab723c61a7dd0f4f: Generated new TSK 0
I20260806 22:53:47.889823 18569 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:53:47.893644 18580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:47.887213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52077 (local address 127.0.0.1:40759)
0806 22:53:47.887249 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0806 22:53:47.887251 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:53:47.887374 (+ 123us) server_negotiation.cc:407] Connection header received
0806 22:53:47.887446 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:53:47.887447 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:53:47.887457 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:53:47.887511 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:53:47.888016 (+ 505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:47.888531 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:47.889248 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:47.889545 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:47.889609 (+ 64us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:47.890748 (+ 1139us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:53:47.890749 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:53:47.890750 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:53:47.890757 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:53:47.892174 (+ 1417us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:53:47.892689 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:53:47.892691 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:53:47.892691 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:53:47.892745 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:53:47.893041 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:53:47.893042 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:53:47.893042 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:53:47.893198 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:53:47.893296 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 22:53:47.893472 (+ 176us) server_negotiation.cc:299] Negotiation successful
0806 22:53:47.893534 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 22:53:47.895732 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 18487
W20260806 22:53:47.901629 18471 connection.cc:535] client connection to 127.0.0.1:40759 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40759): Connection reset by peer (error 104)
2026-08-06T22:53:47Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1349 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786056586788339-29547-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:53:47 dist-test-slave-1xrh krb5kdc[18591](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:53:47 dist-test-slave-1xrh krb5kdc[18591](info): set up 2 sockets
Aug 06 22:53:47 dist-test-slave-1xrh krb5kdc[18591](info): commencing operation
krb5kdc: starting...
W20260806 22:53:48.955091 29547 mini_kdc.cc:121] Time spent starting KDC: real 1.040s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786056586788339-29547-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786056586788339-29547-0/krb5kdc/test-user.keytab.
Aug 06 22:53:49 dist-test-slave-1xrh krb5kdc[18591](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056829, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:53:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:53:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786056586788339-29547-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786056586788339-29547-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 22:53:49.073464 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786056586788339-29547-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786056586788339-29547-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786056586788339-29547-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46395
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786056586788339-29547-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33899
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46395
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:53:49.153137 18604 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:53:49.153359 18604 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:53:49.153398 18604 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:53:49.154592 18604 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:53:49.154661 18604 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:53:49.154685 18604 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:53:49.154706 18604 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:53:49.154736 18604 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:53:49.156205 18604 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33899
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786056586788339-29547-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786056586788339-29547-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46395
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786056586788339-29547-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46395
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18604
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786056586788339-29547-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:53:49.156471 18604 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:53:49.156713 18604 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 22:53:49.161662 18604 server_base.cc:1034] running on GCE node
W20260806 22:53:49.161595 18610 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:53:49.161629 18609 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:53:49.161594 18612 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:53:49.162034 18604 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:53:49.162305 18604 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:53:49.163475 18604 hybrid_clock.cc:648] HybridClock initialized: now 1786056829163471 us; error 36 us; skew 500 ppm
Aug 06 22:53:49 dist-test-slave-1xrh krb5kdc[18591](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056829, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:53:49.165879 18604 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 22:53:49.166339 18604 webserver.cc:460] Webserver started at http://127.0.0.1:35641/ using document root <none> and password file <none>
I20260806 22:53:49.166579 18604 fs_manager.cc:359] Metadata directory not provided
I20260806 22:53:49.166648 18604 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:53:49.166797 18604 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:53:49.167853 18604 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786056586788339-29547-0/minicluster-data/master-0/data/instance:
uuid: "904fd9be2ffc48219cf1801d6da5a961"
format_stamp: "Formatted at 2026-08-06 22:53:49 on dist-test-slave-1xrh"
server_key: "cb259247908c24eab01b88b1d961da2d"
server_key_iv: "392b84f07763b49569d4e60947a1f262"
server_key_version: "encryptionkey@0"
I20260806 22:53:49.168226 18604 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786056586788339-29547-0/minicluster-data/master-0/wal/instance:
uuid: "904fd9be2ffc48219cf1801d6da5a961"
format_stamp: "Formatted at 2026-08-06 22:53:49 on dist-test-slave-1xrh"
server_key: "cb259247908c24eab01b88b1d961da2d"
server_key_iv: "392b84f07763b49569d4e60947a1f262"
server_key_version: "encryptionkey@0"
I20260806 22:53:49.169519 18604 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 22:53:49.170213 18619 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:53:49.170379 18604 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:53:49.170466 18604 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786056586788339-29547-0/minicluster-data/master-0/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786056586788339-29547-0/minicluster-data/master-0/wal
uuid: "904fd9be2ffc48219cf1801d6da5a961"
format_stamp: "Formatted at 2026-08-06 22:53:49 on dist-test-slave-1xrh"
server_key: "cb259247908c24eab01b88b1d961da2d"
server_key_iv: "392b84f07763b49569d4e60947a1f262"
server_key_version: "encryptionkey@0"
I20260806 22:53:49.170548 18604 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786056586788339-29547-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786056586788339-29547-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786056586788339-29547-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:53:49.189152 18604 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:53:49.190891 18604 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:53:49.191051 18604 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:53:49.195065 18604 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46395
I20260806 22:53:49.195153 18676 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46395 every 8 connection(s)
I20260806 22:53:49.195480 18604 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
I20260806 22:53:49.196286 18677 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:53:49.197880 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 18604
I20260806 22:53:49.198000 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786056586788339-29547-0/minicluster-data/master-0/wal/instance
I20260806 22:53:49.198200 29547 external_mini_cluster.cc:1428] Setting key e10fb86dbaa60ec09a31a29bf34bf007
I20260806 22:53:49.198832 18677 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 904fd9be2ffc48219cf1801d6da5a961: Bootstrap starting.
I20260806 22:53:49.199478 18677 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 904fd9be2ffc48219cf1801d6da5a961: Neither blocks nor log segments found. Creating new log.
I20260806 22:53:49.199810 18677 log.cc:824] T 00000000000000000000000000000000 P 904fd9be2ffc48219cf1801d6da5a961: Log is configured to *not* fsync() on all Append() calls
I20260806 22:53:49.200552 18677 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 904fd9be2ffc48219cf1801d6da5a961: No bootstrap required, opened a new log
I20260806 22:53:49.202805 18677 raft_consensus.cc:348] T 00000000000000000000000000000000 P 904fd9be2ffc48219cf1801d6da5a961 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "904fd9be2ffc48219cf1801d6da5a961" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46395 } }
I20260806 22:53:49.202958 18677 raft_consensus.cc:374] T 00000000000000000000000000000000 P 904fd9be2ffc48219cf1801d6da5a961 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:53:49.202988 18677 raft_consensus.cc:729] T 00000000000000000000000000000000 P 904fd9be2ffc48219cf1801d6da5a961 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 904fd9be2ffc48219cf1801d6da5a961, State: Initialized, Role: FOLLOWER
I20260806 22:53:49.203152 18677 consensus_queue.cc:260] T 00000000000000000000000000000000 P 904fd9be2ffc48219cf1801d6da5a961 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "904fd9be2ffc48219cf1801d6da5a961" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46395 } }
I20260806 22:53:49.203244 18677 raft_consensus.cc:388] T 00000000000000000000000000000000 P 904fd9be2ffc48219cf1801d6da5a961 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:53:49.203285 18677 raft_consensus.cc:482] T 00000000000000000000000000000000 P 904fd9be2ffc48219cf1801d6da5a961 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:53:49.203330 18677 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 904fd9be2ffc48219cf1801d6da5a961 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:53:49.204135 18677 raft_consensus.cc:504] T 00000000000000000000000000000000 P 904fd9be2ffc48219cf1801d6da5a961 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "904fd9be2ffc48219cf1801d6da5a961" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46395 } }
I20260806 22:53:49.204206 18680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:49.198669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40329 (local address 127.0.0.1:46395)
0806 22:53:49.198868 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0806 22:53:49.198881 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 22:53:49.198914 (+ 33us) server_negotiation.cc:407] Connection header received
0806 22:53:49.199411 (+ 497us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:53:49.199418 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:53:49.199462 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:53:49.199614 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:53:49.199965 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:49.201020 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:49.202169 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:49.202546 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:49.202622 (+ 76us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:49.203393 (+ 771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:53:49.203398 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:53:49.203398 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:53:49.203623 (+ 225us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:53:49.203671 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:53:49.203687 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:53:49.203774 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 22:53:49.203961 (+ 187us) server_negotiation.cc:299] Negotiation successful
0806 22:53:49.203991 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":98,"threads_started":1}
I20260806 22:53:49.204238 18677 leader_election.cc:302] T 00000000000000000000000000000000 P 904fd9be2ffc48219cf1801d6da5a961 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 904fd9be2ffc48219cf1801d6da5a961; no voters:
I20260806 22:53:49.204389 18677 leader_election.cc:288] T 00000000000000000000000000000000 P 904fd9be2ffc48219cf1801d6da5a961 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:53:49.204450 18682 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 904fd9be2ffc48219cf1801d6da5a961 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:53:49.204602 18682 raft_consensus.cc:686] T 00000000000000000000000000000000 P 904fd9be2ffc48219cf1801d6da5a961 [term 1 LEADER]: Becoming Leader. State: Replica: 904fd9be2ffc48219cf1801d6da5a961, State: Running, Role: LEADER
I20260806 22:53:49.204634 18677 sys_catalog.cc:564] T 00000000000000000000000000000000 P 904fd9be2ffc48219cf1801d6da5a961 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:53:49.204690 18682 consensus_queue.cc:237] T 00000000000000000000000000000000 P 904fd9be2ffc48219cf1801d6da5a961 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "904fd9be2ffc48219cf1801d6da5a961" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46395 } }
I20260806 22:53:49.204981 18682 sys_catalog.cc:455] T 00000000000000000000000000000000 P 904fd9be2ffc48219cf1801d6da5a961 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 904fd9be2ffc48219cf1801d6da5a961. Latest consensus state: current_term: 1 leader_uuid: "904fd9be2ffc48219cf1801d6da5a961" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "904fd9be2ffc48219cf1801d6da5a961" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46395 } } }
I20260806 22:53:49.205090 18682 sys_catalog.cc:458] T 00000000000000000000000000000000 P 904fd9be2ffc48219cf1801d6da5a961 [sys.catalog]: This master's current role is: LEADER
I20260806 22:53:49.205307 29547 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:53:49.205340 18683 sys_catalog.cc:455] T 00000000000000000000000000000000 P 904fd9be2ffc48219cf1801d6da5a961 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "904fd9be2ffc48219cf1801d6da5a961" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "904fd9be2ffc48219cf1801d6da5a961" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46395 } } }
I20260806 22:53:49.205408 18683 sys_catalog.cc:458] T 00000000000000000000000000000000 P 904fd9be2ffc48219cf1801d6da5a961 [sys.catalog]: This master's current role is: LEADER
I20260806 22:53:49.205947 18688 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:53:49.206409 18688 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:53:49.207966 18688 catalog_manager.cc:1347] Generated new cluster ID: 8462f3fe047c49a08f638cfa55f84abb
I20260806 22:53:49.208014 18688 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:53:49.210283 18680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:49.206886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46213 (local address 127.0.0.1:46395)
0806 22:53:49.206915 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0806 22:53:49.206917 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:53:49.207073 (+ 156us) server_negotiation.cc:407] Connection header received
0806 22:53:49.207117 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:53:49.207118 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:53:49.207145 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:53:49.207191 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:53:49.207337 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:49.207999 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:49.209009 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:49.209333 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:49.209389 (+ 56us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:49.210051 (+ 662us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:53:49.210052 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:53:49.210053 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:53:49.210056 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:53:49.210069 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:53:49.210073 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:53:49.210123 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 22:53:49.210211 (+ 88us) server_negotiation.cc:299] Negotiation successful
0806 22:53:49.210220 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260806 22:53:49.229287 18688 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:53:49.229949 18688 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:53:49.235069 18688 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 904fd9be2ffc48219cf1801d6da5a961: Generated new TSK 0
I20260806 22:53:49.235257 18688 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:53:49.239199 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 18604
W20260806 22:53:49.245441 18589 connection.cc:535] client connection to 127.0.0.1:46395 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46395): Connection reset by peer (error 104)
2026-08-06T22:53:49Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1343 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786056586788339-29547-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:53:49 dist-test-slave-1xrh krb5kdc[18709](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:53:49 dist-test-slave-1xrh krb5kdc[18709](info): set up 2 sockets
Aug 06 22:53:49 dist-test-slave-1xrh krb5kdc[18709](info): commencing operation
krb5kdc: starting...
W20260806 22:53:50.290544 29547 mini_kdc.cc:121] Time spent starting KDC: real 1.031s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786056586788339-29547-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786056586788339-29547-0/krb5kdc/test-user.keytab.
Aug 06 22:53:50 dist-test-slave-1xrh krb5kdc[18709](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056830, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:53:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:53:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786056586788339-29547-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786056586788339-29547-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 22:53:50.407553 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786056586788339-29547-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786056586788339-29547-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786056586788339-29547-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35521
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786056586788339-29547-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35521
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:53:50.485502 18724 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:53:50.485688 18724 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:53:50.485710 18724 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:53:50.486728 18724 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:53:50.486760 18724 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:53:50.486773 18724 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:53:50.486785 18724 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:53:50.486797 18724 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:53:50.488111 18724 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786056586788339-29547-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786056586788339-29547-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35521
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786056586788339-29547-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35521
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18724
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786056586788339-29547-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:53:50.488277 18724 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:53:50.488426 18724 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:53:50.492954 18730 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:53:50.492972 18732 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:53:50.493011 18729 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:53:50.493146 18724 server_base.cc:1034] running on GCE node
I20260806 22:53:50.493409 18724 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:53:50.493666 18724 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:53:50.494819 18724 hybrid_clock.cc:648] HybridClock initialized: now 1786056830494803 us; error 27 us; skew 500 ppm
Aug 06 22:53:50 dist-test-slave-1xrh krb5kdc[18709](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056830, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:53:50.497236 18724 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 22:53:50.497633 18724 webserver.cc:460] Webserver started at http://127.0.0.1:44493/ using document root <none> and password file <none>
I20260806 22:53:50.497840 18724 fs_manager.cc:359] Metadata directory not provided
I20260806 22:53:50.497905 18724 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:53:50.498052 18724 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:53:50.499037 18724 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786056586788339-29547-0/minicluster-data/master-0/data/instance:
uuid: "a1823b9626da4681a1561bb868ba9bb5"
format_stamp: "Formatted at 2026-08-06 22:53:50 on dist-test-slave-1xrh"
server_key: "85f41ba9408d8af7fe204c674d65cbb5"
server_key_iv: "b0fddfe85945be4a5a78314240de4944"
server_key_version: "encryptionkey@0"
I20260806 22:53:50.499476 18724 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786056586788339-29547-0/minicluster-data/master-0/wal/instance:
uuid: "a1823b9626da4681a1561bb868ba9bb5"
format_stamp: "Formatted at 2026-08-06 22:53:50 on dist-test-slave-1xrh"
server_key: "85f41ba9408d8af7fe204c674d65cbb5"
server_key_iv: "b0fddfe85945be4a5a78314240de4944"
server_key_version: "encryptionkey@0"
I20260806 22:53:50.500782 18724 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:53:50.501487 18740 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:53:50.501682 18724 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:53:50.501780 18724 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786056586788339-29547-0/minicluster-data/master-0/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786056586788339-29547-0/minicluster-data/master-0/wal
uuid: "a1823b9626da4681a1561bb868ba9bb5"
format_stamp: "Formatted at 2026-08-06 22:53:50 on dist-test-slave-1xrh"
server_key: "85f41ba9408d8af7fe204c674d65cbb5"
server_key_iv: "b0fddfe85945be4a5a78314240de4944"
server_key_version: "encryptionkey@0"
I20260806 22:53:50.501843 18724 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786056586788339-29547-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786056586788339-29547-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786056586788339-29547-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:53:50.505034 18724 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:53:50.507085 18724 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:53:50.507242 18724 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:53:50.510896 18724 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35521
I20260806 22:53:50.510960 18793 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35521 every 8 connection(s)
I20260806 22:53:50.511284 18724 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
I20260806 22:53:50.511621 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 18724
I20260806 22:53:50.511744 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786056586788339-29547-0/minicluster-data/master-0/wal/instance
I20260806 22:53:50.511914 29547 external_mini_cluster.cc:1428] Setting key afde31836aa7a0ddd40a664d674fe19f
I20260806 22:53:50.512409 18794 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:53:50.514464 18794 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a1823b9626da4681a1561bb868ba9bb5: Bootstrap starting.
I20260806 22:53:50.515128 18794 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a1823b9626da4681a1561bb868ba9bb5: Neither blocks nor log segments found. Creating new log.
I20260806 22:53:50.515479 18794 log.cc:824] T 00000000000000000000000000000000 P a1823b9626da4681a1561bb868ba9bb5: Log is configured to *not* fsync() on all Append() calls
I20260806 22:53:50.516198 18794 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a1823b9626da4681a1561bb868ba9bb5: No bootstrap required, opened a new log
I20260806 22:53:50.516716 18797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:50.512424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53239 (local address 127.0.0.1:35521)
0806 22:53:50.514725 (+ 2301us) server_negotiation.cc:206] Beginning negotiation
0806 22:53:50.514733 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:53:50.514768 (+ 35us) server_negotiation.cc:407] Connection header received
0806 22:53:50.515414 (+ 646us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:53:50.515422 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:53:50.515455 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:53:50.515601 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:53:50.516299 (+ 698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:53:50.516303 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:53:50.516303 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:53:50.516308 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:53:50.516354 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:53:50.516369 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:53:50.516439 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 22:53:50.516490 (+ 51us) server_negotiation.cc:299] Negotiation successful
0806 22:53:50.516527 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2239,"thread_start_us":76,"threads_started":1}
I20260806 22:53:50.517620 29547 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:53:50.518450 18794 raft_consensus.cc:348] T 00000000000000000000000000000000 P a1823b9626da4681a1561bb868ba9bb5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1823b9626da4681a1561bb868ba9bb5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35521 } }
I20260806 22:53:50.518620 18794 raft_consensus.cc:374] T 00000000000000000000000000000000 P a1823b9626da4681a1561bb868ba9bb5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:53:50.518658 18794 raft_consensus.cc:729] T 00000000000000000000000000000000 P a1823b9626da4681a1561bb868ba9bb5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1823b9626da4681a1561bb868ba9bb5, State: Initialized, Role: FOLLOWER
I20260806 22:53:50.518790 18794 consensus_queue.cc:260] T 00000000000000000000000000000000 P a1823b9626da4681a1561bb868ba9bb5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1823b9626da4681a1561bb868ba9bb5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35521 } }
I20260806 22:53:50.518895 18794 raft_consensus.cc:388] T 00000000000000000000000000000000 P a1823b9626da4681a1561bb868ba9bb5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:53:50.518937 18794 raft_consensus.cc:482] T 00000000000000000000000000000000 P a1823b9626da4681a1561bb868ba9bb5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:53:50.518990 18794 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a1823b9626da4681a1561bb868ba9bb5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:53:50.519855 18794 raft_consensus.cc:504] T 00000000000000000000000000000000 P a1823b9626da4681a1561bb868ba9bb5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1823b9626da4681a1561bb868ba9bb5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35521 } }
I20260806 22:53:50.519973 18794 leader_election.cc:302] T 00000000000000000000000000000000 P a1823b9626da4681a1561bb868ba9bb5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a1823b9626da4681a1561bb868ba9bb5; no voters:
I20260806 22:53:50.520143 18794 leader_election.cc:288] T 00000000000000000000000000000000 P a1823b9626da4681a1561bb868ba9bb5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:53:50.520162 18805 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a1823b9626da4681a1561bb868ba9bb5 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:53:50.520251 18797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:50.519183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44755 (local address 127.0.0.1:35521)
0806 22:53:50.519221 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0806 22:53:50.519222 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:53:50.519317 (+ 95us) server_negotiation.cc:407] Connection header received
0806 22:53:50.519389 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:53:50.519390 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:53:50.519393 (+ 3us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:53:50.519431 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:53:50.519912 (+ 481us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:53:50.519913 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:53:50.519913 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:53:50.519917 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:53:50.519949 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:53:50.519954 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:53:50.520009 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 22:53:50.520125 (+ 116us) server_negotiation.cc:299] Negotiation successful
0806 22:53:50.520137 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 22:53:50.520403 18794 sys_catalog.cc:564] T 00000000000000000000000000000000 P a1823b9626da4681a1561bb868ba9bb5 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:53:50.520452 18805 raft_consensus.cc:686] T 00000000000000000000000000000000 P a1823b9626da4681a1561bb868ba9bb5 [term 1 LEADER]: Becoming Leader. State: Replica: a1823b9626da4681a1561bb868ba9bb5, State: Running, Role: LEADER
I20260806 22:53:50.520673 18805 consensus_queue.cc:237] T 00000000000000000000000000000000 P a1823b9626da4681a1561bb868ba9bb5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1823b9626da4681a1561bb868ba9bb5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35521 } }
I20260806 22:53:50.521046 18806 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1823b9626da4681a1561bb868ba9bb5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a1823b9626da4681a1561bb868ba9bb5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1823b9626da4681a1561bb868ba9bb5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35521 } } }
I20260806 22:53:50.521080 18807 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1823b9626da4681a1561bb868ba9bb5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a1823b9626da4681a1561bb868ba9bb5. Latest consensus state: current_term: 1 leader_uuid: "a1823b9626da4681a1561bb868ba9bb5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1823b9626da4681a1561bb868ba9bb5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35521 } } }
I20260806 22:53:50.521210 18807 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1823b9626da4681a1561bb868ba9bb5 [sys.catalog]: This master's current role is: LEADER
I20260806 22:53:50.521406 18806 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1823b9626da4681a1561bb868ba9bb5 [sys.catalog]: This master's current role is: LEADER
I20260806 22:53:50.521610 18815 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:53:50.522123 18815 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:53:50.523641 18815 catalog_manager.cc:1347] Generated new cluster ID: 7af908fb697a42e692ea999273229ea5
I20260806 22:53:50.523690 18815 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:53:50.530794 18815 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:53:50.530936 18815 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:53:50.541536 18815 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a1823b9626da4681a1561bb868ba9bb5: Generated new TSK 0
I20260806 22:53:50.541708 18815 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:53:50.547523 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 18724
2026-08-06T22:53:50Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1307 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786056586788339-29547-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:53:50 dist-test-slave-1xrh krb5kdc[18828](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:53:50 dist-test-slave-1xrh krb5kdc[18828](info): set up 2 sockets
Aug 06 22:53:50 dist-test-slave-1xrh krb5kdc[18828](info): commencing operation
krb5kdc: starting...
W20260806 22:53:51.591626 29547 mini_kdc.cc:121] Time spent starting KDC: real 1.025s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786056586788339-29547-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786056586788339-29547-0/krb5kdc/test-user.keytab.
Aug 06 22:53:51 dist-test-slave-1xrh krb5kdc[18828](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056831, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:53:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:53:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786056586788339-29547-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786056586788339-29547-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 22:53:51.710357 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786056586788339-29547-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786056586788339-29547-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786056586788339-29547-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45185
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786056586788339-29547-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45185
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:53:51.792687 18841 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:53:51.792896 18841 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:53:51.792918 18841 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:53:51.793989 18841 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:53:51.794025 18841 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:53:51.794044 18841 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:53:51.794056 18841 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:53:51.794068 18841 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:53:51.795593 18841 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786056586788339-29547-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786056586788339-29547-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45185
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786056586788339-29547-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45185
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18841
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786056586788339-29547-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:53:51.795849 18841 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:53:51.796087 18841 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:53:51.800824 18849 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:53:51.800956 18847 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:53:51.800981 18846 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:53:51.801216 18841 server_base.cc:1034] running on GCE node
I20260806 22:53:51.801442 18841 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:53:51.801718 18841 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:53:51.802891 18841 hybrid_clock.cc:648] HybridClock initialized: now 1786056831802885 us; error 35 us; skew 500 ppm
Aug 06 22:53:51 dist-test-slave-1xrh krb5kdc[18828](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056831, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:53:51.805362 18841 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 22:53:51.805730 18841 webserver.cc:460] Webserver started at http://127.0.0.1:36977/ using document root <none> and password file <none>
I20260806 22:53:51.805917 18841 fs_manager.cc:359] Metadata directory not provided
I20260806 22:53:51.805958 18841 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:53:51.806051 18841 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:53:51.807004 18841 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786056586788339-29547-0/minicluster-data/master-0/data/instance:
uuid: "e8823760b0374f58b30c7342b29cf4c7"
format_stamp: "Formatted at 2026-08-06 22:53:51 on dist-test-slave-1xrh"
server_key: "aafed818baf91ddf4de20c1b3d28c230"
server_key_iv: "1212b727723f6c8a8e1e43d1f632d8af"
server_key_version: "encryptionkey@0"
I20260806 22:53:51.807384 18841 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786056586788339-29547-0/minicluster-data/master-0/wal/instance:
uuid: "e8823760b0374f58b30c7342b29cf4c7"
format_stamp: "Formatted at 2026-08-06 22:53:51 on dist-test-slave-1xrh"
server_key: "aafed818baf91ddf4de20c1b3d28c230"
server_key_iv: "1212b727723f6c8a8e1e43d1f632d8af"
server_key_version: "encryptionkey@0"
I20260806 22:53:51.808611 18841 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:53:51.809262 18856 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:53:51.809412 18841 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 22:53:51.809509 18841 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786056586788339-29547-0/minicluster-data/master-0/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786056586788339-29547-0/minicluster-data/master-0/wal
uuid: "e8823760b0374f58b30c7342b29cf4c7"
format_stamp: "Formatted at 2026-08-06 22:53:51 on dist-test-slave-1xrh"
server_key: "aafed818baf91ddf4de20c1b3d28c230"
server_key_iv: "1212b727723f6c8a8e1e43d1f632d8af"
server_key_version: "encryptionkey@0"
I20260806 22:53:51.809563 18841 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786056586788339-29547-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786056586788339-29547-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786056586788339-29547-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:53:51.820248 18841 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:53:51.822158 18841 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:53:51.822326 18841 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:53:51.826426 18841 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45185
I20260806 22:53:51.826488 18910 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45185 every 8 connection(s)
I20260806 22:53:51.826830 18841 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
I20260806 22:53:51.827698 18911 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:53:51.829851 18911 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8823760b0374f58b30c7342b29cf4c7: Bootstrap starting.
I20260806 22:53:51.830410 18911 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e8823760b0374f58b30c7342b29cf4c7: Neither blocks nor log segments found. Creating new log.
I20260806 22:53:51.830704 18911 log.cc:824] T 00000000000000000000000000000000 P e8823760b0374f58b30c7342b29cf4c7: Log is configured to *not* fsync() on all Append() calls
I20260806 22:53:51.831424 18911 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8823760b0374f58b30c7342b29cf4c7: No bootstrap required, opened a new log
I20260806 22:53:51.832975 18911 raft_consensus.cc:348] T 00000000000000000000000000000000 P e8823760b0374f58b30c7342b29cf4c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8823760b0374f58b30c7342b29cf4c7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45185 } }
I20260806 22:53:51.833107 18911 raft_consensus.cc:374] T 00000000000000000000000000000000 P e8823760b0374f58b30c7342b29cf4c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:53:51.833163 18911 raft_consensus.cc:729] T 00000000000000000000000000000000 P e8823760b0374f58b30c7342b29cf4c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8823760b0374f58b30c7342b29cf4c7, State: Initialized, Role: FOLLOWER
I20260806 22:53:51.833328 18911 consensus_queue.cc:260] T 00000000000000000000000000000000 P e8823760b0374f58b30c7342b29cf4c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8823760b0374f58b30c7342b29cf4c7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45185 } }
I20260806 22:53:51.833389 18911 raft_consensus.cc:388] T 00000000000000000000000000000000 P e8823760b0374f58b30c7342b29cf4c7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:53:51.833428 18911 raft_consensus.cc:482] T 00000000000000000000000000000000 P e8823760b0374f58b30c7342b29cf4c7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:53:51.833477 18911 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e8823760b0374f58b30c7342b29cf4c7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:53:51.834002 18911 raft_consensus.cc:504] T 00000000000000000000000000000000 P e8823760b0374f58b30c7342b29cf4c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8823760b0374f58b30c7342b29cf4c7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45185 } }
I20260806 22:53:51.834112 18911 leader_election.cc:302] T 00000000000000000000000000000000 P e8823760b0374f58b30c7342b29cf4c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e8823760b0374f58b30c7342b29cf4c7; no voters:
I20260806 22:53:51.834297 18911 leader_election.cc:288] T 00000000000000000000000000000000 P e8823760b0374f58b30c7342b29cf4c7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:53:51.834334 18914 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e8823760b0374f58b30c7342b29cf4c7 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:53:51.834534 18914 raft_consensus.cc:686] T 00000000000000000000000000000000 P e8823760b0374f58b30c7342b29cf4c7 [term 1 LEADER]: Becoming Leader. State: Replica: e8823760b0374f58b30c7342b29cf4c7, State: Running, Role: LEADER
I20260806 22:53:51.834620 18914 consensus_queue.cc:237] T 00000000000000000000000000000000 P e8823760b0374f58b30c7342b29cf4c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8823760b0374f58b30c7342b29cf4c7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45185 } }
I20260806 22:53:51.834719 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 18841
I20260806 22:53:51.834846 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786056586788339-29547-0/minicluster-data/master-0/wal/instance
I20260806 22:53:51.834997 18916 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8823760b0374f58b30c7342b29cf4c7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e8823760b0374f58b30c7342b29cf4c7. Latest consensus state: current_term: 1 leader_uuid: "e8823760b0374f58b30c7342b29cf4c7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8823760b0374f58b30c7342b29cf4c7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45185 } } }
I20260806 22:53:51.835070 29547 external_mini_cluster.cc:1428] Setting key 80d4f23290d337f567c826311702e81a
I20260806 22:53:51.835085 18916 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8823760b0374f58b30c7342b29cf4c7 [sys.catalog]: This master's current role is: LEADER
I20260806 22:53:51.835210 18915 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8823760b0374f58b30c7342b29cf4c7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e8823760b0374f58b30c7342b29cf4c7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8823760b0374f58b30c7342b29cf4c7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45185 } } }
I20260806 22:53:51.835285 18915 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8823760b0374f58b30c7342b29cf4c7 [sys.catalog]: This master's current role is: LEADER
I20260806 22:53:51.835320 18911 sys_catalog.cc:564] T 00000000000000000000000000000000 P e8823760b0374f58b30c7342b29cf4c7 [sys.catalog]: configured and running, proceeding with master startup.
W20260806 22:53:51.837194 18932 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e8823760b0374f58b30c7342b29cf4c7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 22:53:51.837252 18932 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 22:53:51.837314 18918 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:53:51.837496 18918 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:53:51.838907 18918 catalog_manager.cc:1347] Generated new cluster ID: a55b90da7c264e77a1a420afd1890236
I20260806 22:53:51.838976 18918 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 22:53:51 dist-test-slave-1xrh krb5kdc[18828](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056831, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 22:53:51.844552 18918 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:53:51.845019 18918 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:53:51.848451 18918 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e8823760b0374f58b30c7342b29cf4c7: Generated new TSK 0
I20260806 22:53:51.848640 18923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:51.835715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43383 (local address 127.0.0.1:45185)
0806 22:53:51.835928 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0806 22:53:51.835937 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:53:51.835972 (+ 35us) server_negotiation.cc:407] Connection header received
0806 22:53:51.836143 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:53:51.836148 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:53:51.836189 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:53:51.836299 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:53:51.837149 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:51.838336 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:51.839438 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:51.839851 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:51.839979 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:51.842320 (+ 2341us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:53:51.842324 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:53:51.842324 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:53:51.842336 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:53:51.844206 (+ 1870us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:53:51.847265 (+ 3059us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:53:51.847269 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:53:51.847269 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:53:51.847392 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:53:51.847604 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:53:51.847605 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:53:51.847606 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:53:51.847878 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:53:51.848057 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0806 22:53:51.848266 (+ 209us) server_negotiation.cc:299] Negotiation successful
0806 22:53:51.848416 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":121,"threads_started":1}
I20260806 22:53:51.848642 18918 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:53:51.849583 29547 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:53:51.857326 18923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:51.851093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54087 (local address 127.0.0.1:45185)
0806 22:53:51.851152 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0806 22:53:51.851153 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:53:51.851242 (+ 89us) server_negotiation.cc:407] Connection header received
0806 22:53:51.851301 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:53:51.851302 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:53:51.851311 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:53:51.851353 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:53:51.851969 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:51.852415 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:51.853463 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:51.853667 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:51.853715 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:51.854967 (+ 1252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:53:51.854969 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:53:51.854969 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:53:51.854976 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:53:51.856327 (+ 1351us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:53:51.856697 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:53:51.856697 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:53:51.856698 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:53:51.856745 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:53:51.856930 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:53:51.856930 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:53:51.856930 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:53:51.857017 (+ 87us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:53:51.857093 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 22:53:51.857216 (+ 123us) server_negotiation.cc:299] Negotiation successful
0806 22:53:51.857273 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 22:53:51.859242 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 18841
2026-08-06T22:53:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1312 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786056586788339-29547-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:53:51 dist-test-slave-1xrh krb5kdc[18943](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:53:51 dist-test-slave-1xrh krb5kdc[18943](info): set up 2 sockets
Aug 06 22:53:51 dist-test-slave-1xrh krb5kdc[18943](info): commencing operation
krb5kdc: starting...
W20260806 22:53:52.910992 29547 mini_kdc.cc:121] Time spent starting KDC: real 1.032s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786056586788339-29547-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786056586788339-29547-0/krb5kdc/test-user.keytab.
Aug 06 22:53:52 dist-test-slave-1xrh krb5kdc[18943](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056832, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:53:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:53:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786056586788339-29547-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786056586788339-29547-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 22:53:53.028102 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786056586788339-29547-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786056586788339-29547-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786056586788339-29547-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42435
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786056586788339-29547-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42435
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:53:53.107200 18958 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:53:53.107389 18958 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:53:53.107412 18958 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:53:53.108522 18958 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:53:53.108557 18958 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:53:53.108572 18958 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:53:53.108584 18958 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:53:53.108596 18958 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:53:53.109953 18958 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786056586788339-29547-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786056586788339-29547-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42435
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786056586788339-29547-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42435
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18958
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786056586788339-29547-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:53:53.110121 18958 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:53:53.110270 18958 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:53:53.114893 18966 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:53:53.114945 18965 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:53:53.115027 18958 server_base.cc:1034] running on GCE node
W20260806 22:53:53.114902 18968 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:53:53.115383 18958 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:53:53.115657 18958 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:53:53.116823 18958 hybrid_clock.cc:648] HybridClock initialized: now 1786056833116797 us; error 53 us; skew 500 ppm
Aug 06 22:53:53 dist-test-slave-1xrh krb5kdc[18943](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056833, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:53:53.124527 18958 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 22:53:53.124939 18958 webserver.cc:460] Webserver started at http://127.0.0.1:34895/ using document root <none> and password file <none>
I20260806 22:53:53.125174 18958 fs_manager.cc:359] Metadata directory not provided
I20260806 22:53:53.125238 18958 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:53:53.125376 18958 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:53:53.126331 18958 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786056586788339-29547-0/minicluster-data/master-0/data/instance:
uuid: "de24dfd915fc44c4a38ef9a8d2128a7a"
format_stamp: "Formatted at 2026-08-06 22:53:53 on dist-test-slave-1xrh"
server_key: "3c43a62884fea04462c8de263d71842b"
server_key_iv: "fce1bd87ee8d201bc3ca5427cd216b73"
server_key_version: "encryptionkey@0"
I20260806 22:53:53.126677 18958 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786056586788339-29547-0/minicluster-data/master-0/wal/instance:
uuid: "de24dfd915fc44c4a38ef9a8d2128a7a"
format_stamp: "Formatted at 2026-08-06 22:53:53 on dist-test-slave-1xrh"
server_key: "3c43a62884fea04462c8de263d71842b"
server_key_iv: "fce1bd87ee8d201bc3ca5427cd216b73"
server_key_version: "encryptionkey@0"
I20260806 22:53:53.128042 18958 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260806 22:53:53.128755 18975 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:53:53.128918 18958 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:53:53.129029 18958 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786056586788339-29547-0/minicluster-data/master-0/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786056586788339-29547-0/minicluster-data/master-0/wal
uuid: "de24dfd915fc44c4a38ef9a8d2128a7a"
format_stamp: "Formatted at 2026-08-06 22:53:53 on dist-test-slave-1xrh"
server_key: "3c43a62884fea04462c8de263d71842b"
server_key_iv: "fce1bd87ee8d201bc3ca5427cd216b73"
server_key_version: "encryptionkey@0"
I20260806 22:53:53.129107 18958 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786056586788339-29547-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786056586788339-29547-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786056586788339-29547-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:53:53.142608 18958 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:53:53.144820 18958 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:53:53.144987 18958 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:53:53.149178 18958 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42435
I20260806 22:53:53.149210 19029 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42435 every 8 connection(s)
I20260806 22:53:53.149574 18958 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
I20260806 22:53:53.150368 19030 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:53:53.152405 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 18958
I20260806 22:53:53.152529 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786056586788339-29547-0/minicluster-data/master-0/wal/instance
I20260806 22:53:53.152738 29547 external_mini_cluster.cc:1428] Setting key 16698c02aed48a6e48e2f40c175bae01
I20260806 22:53:53.152740 19030 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de24dfd915fc44c4a38ef9a8d2128a7a: Bootstrap starting.
I20260806 22:53:53.153430 19030 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P de24dfd915fc44c4a38ef9a8d2128a7a: Neither blocks nor log segments found. Creating new log.
I20260806 22:53:53.153759 19030 log.cc:824] T 00000000000000000000000000000000 P de24dfd915fc44c4a38ef9a8d2128a7a: Log is configured to *not* fsync() on all Append() calls
I20260806 22:53:53.154579 19030 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de24dfd915fc44c4a38ef9a8d2128a7a: No bootstrap required, opened a new log
I20260806 22:53:53.156747 19030 raft_consensus.cc:348] T 00000000000000000000000000000000 P de24dfd915fc44c4a38ef9a8d2128a7a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de24dfd915fc44c4a38ef9a8d2128a7a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42435 } }
I20260806 22:53:53.156877 19030 raft_consensus.cc:374] T 00000000000000000000000000000000 P de24dfd915fc44c4a38ef9a8d2128a7a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:53:53.156898 19030 raft_consensus.cc:729] T 00000000000000000000000000000000 P de24dfd915fc44c4a38ef9a8d2128a7a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de24dfd915fc44c4a38ef9a8d2128a7a, State: Initialized, Role: FOLLOWER
I20260806 22:53:53.157030 19030 consensus_queue.cc:260] T 00000000000000000000000000000000 P de24dfd915fc44c4a38ef9a8d2128a7a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de24dfd915fc44c4a38ef9a8d2128a7a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42435 } }
I20260806 22:53:53.157091 19030 raft_consensus.cc:388] T 00000000000000000000000000000000 P de24dfd915fc44c4a38ef9a8d2128a7a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:53:53.157111 19030 raft_consensus.cc:482] T 00000000000000000000000000000000 P de24dfd915fc44c4a38ef9a8d2128a7a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:53:53.157176 19030 raft_consensus.cc:3037] T 00000000000000000000000000000000 P de24dfd915fc44c4a38ef9a8d2128a7a [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:53:53.157958 19030 raft_consensus.cc:504] T 00000000000000000000000000000000 P de24dfd915fc44c4a38ef9a8d2128a7a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de24dfd915fc44c4a38ef9a8d2128a7a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42435 } }
I20260806 22:53:53.158057 19030 leader_election.cc:302] T 00000000000000000000000000000000 P de24dfd915fc44c4a38ef9a8d2128a7a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: de24dfd915fc44c4a38ef9a8d2128a7a; no voters:
I20260806 22:53:53.158250 19030 leader_election.cc:288] T 00000000000000000000000000000000 P de24dfd915fc44c4a38ef9a8d2128a7a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:53:53.158273 19035 raft_consensus.cc:2781] T 00000000000000000000000000000000 P de24dfd915fc44c4a38ef9a8d2128a7a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:53:53.158502 19035 raft_consensus.cc:686] T 00000000000000000000000000000000 P de24dfd915fc44c4a38ef9a8d2128a7a [term 1 LEADER]: Becoming Leader. State: Replica: de24dfd915fc44c4a38ef9a8d2128a7a, State: Running, Role: LEADER
I20260806 22:53:53.158594 19030 sys_catalog.cc:564] T 00000000000000000000000000000000 P de24dfd915fc44c4a38ef9a8d2128a7a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:53:53.158640 19035 consensus_queue.cc:237] T 00000000000000000000000000000000 P de24dfd915fc44c4a38ef9a8d2128a7a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de24dfd915fc44c4a38ef9a8d2128a7a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42435 } }
I20260806 22:53:53.158607 19033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:53.153260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57283 (local address 127.0.0.1:42435)
0806 22:53:53.153440 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0806 22:53:53.153447 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:53:53.153473 (+ 26us) server_negotiation.cc:407] Connection header received
0806 22:53:53.154055 (+ 582us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:53:53.154064 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:53:53.154126 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:53:53.154276 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:53:53.154474 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:53.155469 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:53.156599 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:53.157069 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:53.157190 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:53.157848 (+ 658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:53:53.157851 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:53:53.157852 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:53:53.158057 (+ 205us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:53:53.158099 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:53:53.158113 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:53:53.158175 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 22:53:53.158304 (+ 129us) server_negotiation.cc:299] Negotiation successful
0806 22:53:53.158375 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":84,"threads_started":1}
I20260806 22:53:53.159090 19038 sys_catalog.cc:455] T 00000000000000000000000000000000 P de24dfd915fc44c4a38ef9a8d2128a7a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "de24dfd915fc44c4a38ef9a8d2128a7a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de24dfd915fc44c4a38ef9a8d2128a7a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42435 } } }
I20260806 22:53:53.159545 19038 sys_catalog.cc:458] T 00000000000000000000000000000000 P de24dfd915fc44c4a38ef9a8d2128a7a [sys.catalog]: This master's current role is: LEADER
I20260806 22:53:53.159098 19037 sys_catalog.cc:455] T 00000000000000000000000000000000 P de24dfd915fc44c4a38ef9a8d2128a7a [sys.catalog]: SysCatalogTable state changed. Reason: New leader de24dfd915fc44c4a38ef9a8d2128a7a. Latest consensus state: current_term: 1 leader_uuid: "de24dfd915fc44c4a38ef9a8d2128a7a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de24dfd915fc44c4a38ef9a8d2128a7a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42435 } } }
I20260806 22:53:53.159687 19037 sys_catalog.cc:458] T 00000000000000000000000000000000 P de24dfd915fc44c4a38ef9a8d2128a7a [sys.catalog]: This master's current role is: LEADER
I20260806 22:53:53.160006 19046 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:53:53.160318 29547 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:53:53.160621 19046 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:53:53.162006 19046 catalog_manager.cc:1347] Generated new cluster ID: bc696b5f88c54e409332aade5976759c
I20260806 22:53:53.162062 19046 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:53:53.169869 19033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:53.167213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43219 (local address 127.0.0.1:42435)
0806 22:53:53.167259 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 22:53:53.167261 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:53:53.167268 (+ 7us) server_negotiation.cc:407] Connection header received
0806 22:53:53.167286 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:53:53.167286 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:53:53.167295 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:53:53.167362 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:53:53.167538 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:53.167991 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:53.168688 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:53.168901 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:53.168973 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:53.169640 (+ 667us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:53:53.169641 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:53:53.169641 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:53:53.169645 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:53:53.169658 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:53:53.169662 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:53:53.169703 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0806 22:53:53.169763 (+ 60us) server_negotiation.cc:299] Negotiation successful
0806 22:53:53.169783 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260806 22:53:53.173655 19046 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:53:53.174281 19046 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:53:53.179538 19046 catalog_manager.cc:5868] T 00000000000000000000000000000000 P de24dfd915fc44c4a38ef9a8d2128a7a: Generated new TSK 0
I20260806 22:53:53.179687 19046 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:53:53.181948 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 18958
2026-08-06T22:53:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1322 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786056586788339-29547-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:53:53 dist-test-slave-1xrh krb5kdc[19062](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:53:53 dist-test-slave-1xrh krb5kdc[19062](info): set up 2 sockets
Aug 06 22:53:53 dist-test-slave-1xrh krb5kdc[19062](info): commencing operation
krb5kdc: starting...
W20260806 22:53:54.230238 29547 mini_kdc.cc:121] Time spent starting KDC: real 1.029s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786056586788339-29547-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786056586788339-29547-0/krb5kdc/test-user.keytab.
Aug 06 22:53:54 dist-test-slave-1xrh krb5kdc[19062](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056834, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:53:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:53:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786056586788339-29547-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786056586788339-29547-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 22:53:54.345417 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786056586788339-29547-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786056586788339-29547-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786056586788339-29547-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33467
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786056586788339-29547-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42491
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33467
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:53:54.424871 19075 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:53:54.425107 19075 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:53:54.425143 19075 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:53:54.426223 19075 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:53:54.426290 19075 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:53:54.426313 19075 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:53:54.426335 19075 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:53:54.426363 19075 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:53:54.427794 19075 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42491
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786056586788339-29547-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786056586788339-29547-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33467
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786056586788339-29547-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33467
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19075
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786056586788339-29547-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:53:54.428048 19075 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:53:54.428264 19075 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:53:54.432722 19080 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:53:54.432703 19081 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:53:54.432684 19084 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:53:54.432860 19075 server_base.cc:1034] running on GCE node
I20260806 22:53:54.433145 19075 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:53:54.433418 19075 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:53:54.434592 19075 hybrid_clock.cc:648] HybridClock initialized: now 1786056834434570 us; error 52 us; skew 500 ppm
Aug 06 22:53:54 dist-test-slave-1xrh krb5kdc[19062](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056834, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:53:54.436978 19075 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 22:53:54.437408 19075 webserver.cc:460] Webserver started at http://127.0.0.1:44631/ using document root <none> and password file <none>
I20260806 22:53:54.437634 19075 fs_manager.cc:359] Metadata directory not provided
I20260806 22:53:54.437702 19075 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:53:54.437852 19075 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:53:54.438817 19075 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786056586788339-29547-0/minicluster-data/master-0/data/instance:
uuid: "ae93cf05bb5e4917bb078d6f9438028e"
format_stamp: "Formatted at 2026-08-06 22:53:54 on dist-test-slave-1xrh"
server_key: "87fa04205ce4d0d88a399e045da68a02"
server_key_iv: "c92335f2227be2469691cf936bd39583"
server_key_version: "encryptionkey@0"
I20260806 22:53:54.439230 19075 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786056586788339-29547-0/minicluster-data/master-0/wal/instance:
uuid: "ae93cf05bb5e4917bb078d6f9438028e"
format_stamp: "Formatted at 2026-08-06 22:53:54 on dist-test-slave-1xrh"
server_key: "87fa04205ce4d0d88a399e045da68a02"
server_key_iv: "c92335f2227be2469691cf936bd39583"
server_key_version: "encryptionkey@0"
I20260806 22:53:54.440477 19075 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:53:54.441183 19092 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:53:54.441349 19075 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:53:54.441443 19075 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786056586788339-29547-0/minicluster-data/master-0/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786056586788339-29547-0/minicluster-data/master-0/wal
uuid: "ae93cf05bb5e4917bb078d6f9438028e"
format_stamp: "Formatted at 2026-08-06 22:53:54 on dist-test-slave-1xrh"
server_key: "87fa04205ce4d0d88a399e045da68a02"
server_key_iv: "c92335f2227be2469691cf936bd39583"
server_key_version: "encryptionkey@0"
I20260806 22:53:54.441529 19075 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786056586788339-29547-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786056586788339-29547-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786056586788339-29547-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:53:54.444772 19075 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:53:54.446890 19075 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:53:54.447043 19075 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:53:54.450863 19075 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33467
I20260806 22:53:54.450937 19146 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33467 every 8 connection(s)
I20260806 22:53:54.451265 19075 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
I20260806 22:53:54.452170 19147 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:53:54.454229 19147 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae93cf05bb5e4917bb078d6f9438028e: Bootstrap starting.
I20260806 22:53:54.454766 19147 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ae93cf05bb5e4917bb078d6f9438028e: Neither blocks nor log segments found. Creating new log.
I20260806 22:53:54.455049 19147 log.cc:824] T 00000000000000000000000000000000 P ae93cf05bb5e4917bb078d6f9438028e: Log is configured to *not* fsync() on all Append() calls
I20260806 22:53:54.455775 19147 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae93cf05bb5e4917bb078d6f9438028e: No bootstrap required, opened a new log
I20260806 22:53:54.457266 19147 raft_consensus.cc:348] T 00000000000000000000000000000000 P ae93cf05bb5e4917bb078d6f9438028e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae93cf05bb5e4917bb078d6f9438028e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33467 } }
I20260806 22:53:54.457387 19147 raft_consensus.cc:374] T 00000000000000000000000000000000 P ae93cf05bb5e4917bb078d6f9438028e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:53:54.457433 19147 raft_consensus.cc:729] T 00000000000000000000000000000000 P ae93cf05bb5e4917bb078d6f9438028e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae93cf05bb5e4917bb078d6f9438028e, State: Initialized, Role: FOLLOWER
I20260806 22:53:54.457602 19147 consensus_queue.cc:260] T 00000000000000000000000000000000 P ae93cf05bb5e4917bb078d6f9438028e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae93cf05bb5e4917bb078d6f9438028e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33467 } }
I20260806 22:53:54.457675 19147 raft_consensus.cc:388] T 00000000000000000000000000000000 P ae93cf05bb5e4917bb078d6f9438028e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:53:54.457715 19147 raft_consensus.cc:482] T 00000000000000000000000000000000 P ae93cf05bb5e4917bb078d6f9438028e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:53:54.457763 19147 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ae93cf05bb5e4917bb078d6f9438028e [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:53:54.458250 19147 raft_consensus.cc:504] T 00000000000000000000000000000000 P ae93cf05bb5e4917bb078d6f9438028e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae93cf05bb5e4917bb078d6f9438028e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33467 } }
I20260806 22:53:54.458349 19147 leader_election.cc:302] T 00000000000000000000000000000000 P ae93cf05bb5e4917bb078d6f9438028e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ae93cf05bb5e4917bb078d6f9438028e; no voters:
I20260806 22:53:54.458519 19147 leader_election.cc:288] T 00000000000000000000000000000000 P ae93cf05bb5e4917bb078d6f9438028e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:53:54.458570 19150 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ae93cf05bb5e4917bb078d6f9438028e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:53:54.458747 19150 raft_consensus.cc:686] T 00000000000000000000000000000000 P ae93cf05bb5e4917bb078d6f9438028e [term 1 LEADER]: Becoming Leader. State: Replica: ae93cf05bb5e4917bb078d6f9438028e, State: Running, Role: LEADER
I20260806 22:53:54.458746 19147 sys_catalog.cc:564] T 00000000000000000000000000000000 P ae93cf05bb5e4917bb078d6f9438028e [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:53:54.458877 19150 consensus_queue.cc:237] T 00000000000000000000000000000000 P ae93cf05bb5e4917bb078d6f9438028e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae93cf05bb5e4917bb078d6f9438028e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33467 } }
I20260806 22:53:54.459311 19152 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae93cf05bb5e4917bb078d6f9438028e [sys.catalog]: SysCatalogTable state changed. Reason: New leader ae93cf05bb5e4917bb078d6f9438028e. Latest consensus state: current_term: 1 leader_uuid: "ae93cf05bb5e4917bb078d6f9438028e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae93cf05bb5e4917bb078d6f9438028e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33467 } } }
I20260806 22:53:54.459394 19152 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae93cf05bb5e4917bb078d6f9438028e [sys.catalog]: This master's current role is: LEADER
I20260806 22:53:54.459322 19151 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae93cf05bb5e4917bb078d6f9438028e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ae93cf05bb5e4917bb078d6f9438028e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae93cf05bb5e4917bb078d6f9438028e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33467 } } }
I20260806 22:53:54.459468 19151 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae93cf05bb5e4917bb078d6f9438028e [sys.catalog]: This master's current role is: LEADER
I20260806 22:53:54.459844 19154 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:53:54.459900 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 19075
I20260806 22:53:54.460031 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786056586788339-29547-0/minicluster-data/master-0/wal/instance
I20260806 22:53:54.460242 29547 external_mini_cluster.cc:1428] Setting key add02e0a76cefaf2a013b42e778ca028
I20260806 22:53:54.460325 19154 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:53:54.461790 19154 catalog_manager.cc:1347] Generated new cluster ID: db346a7b72924709aab207fa9ec74117
I20260806 22:53:54.461851 19154 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:53:54.464840 19169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:54.460820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42151 (local address 127.0.0.1:33467)
0806 22:53:54.463448 (+ 2628us) server_negotiation.cc:206] Beginning negotiation
0806 22:53:54.463454 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:53:54.463484 (+ 30us) server_negotiation.cc:407] Connection header received
0806 22:53:54.463573 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:53:54.463577 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:53:54.463598 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:53:54.463723 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:53:54.464401 (+ 678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:53:54.464405 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:53:54.464405 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:53:54.464409 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:53:54.464454 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:53:54.464465 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:53:54.464540 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 22:53:54.464611 (+ 71us) server_negotiation.cc:299] Negotiation successful
0806 22:53:54.464653 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2573,"thread_start_us":73,"threads_started":1}
I20260806 22:53:54.465691 29547 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:53:54.468402 19169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:54.467366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43079 (local address 127.0.0.1:33467)
0806 22:53:54.467421 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0806 22:53:54.467423 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:53:54.467430 (+ 7us) server_negotiation.cc:407] Connection header received
0806 22:53:54.467448 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:53:54.467449 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:53:54.467452 (+ 3us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:53:54.467537 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:53:54.468182 (+ 645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:53:54.468183 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:53:54.468183 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:53:54.468188 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:53:54.468200 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:53:54.468204 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:53:54.468249 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0806 22:53:54.468308 (+ 59us) server_negotiation.cc:299] Negotiation successful
0806 22:53:54.468321 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 22:53:54.471560 19154 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:53:54.471698 19154 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:53:54.475958 19154 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ae93cf05bb5e4917bb078d6f9438028e: Generated new TSK 0
I20260806 22:53:54.476104 19154 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:53:54.477960 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 19075
2026-08-06T22:53:54Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1296 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786056586788339-29547-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:53:54 dist-test-slave-1xrh krb5kdc[19180](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:53:54 dist-test-slave-1xrh krb5kdc[19180](info): set up 2 sockets
Aug 06 22:53:54 dist-test-slave-1xrh krb5kdc[19180](info): commencing operation
krb5kdc: starting...
W20260806 22:53:55.526789 29547 mini_kdc.cc:121] Time spent starting KDC: real 1.029s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786056586788339-29547-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786056586788339-29547-0/krb5kdc/test-user.keytab.
Aug 06 22:53:55 dist-test-slave-1xrh krb5kdc[19180](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056835, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:53:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:53:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786056586788339-29547-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786056586788339-29547-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 22:53:55.646040 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786056586788339-29547-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786056586788339-29547-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786056586788339-29547-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37319
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786056586788339-29547-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37319
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:53:55.725776 19193 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:53:55.725965 19193 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:53:55.725986 19193 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:53:55.727237 19193 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:53:55.727288 19193 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:53:55.727303 19193 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:53:55.727319 19193 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:53:55.727331 19193 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:53:55.728868 19193 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786056586788339-29547-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786056586788339-29547-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37319
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786056586788339-29547-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37319
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19193
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786056586788339-29547-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:53:55.729048 19193 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:53:55.729206 19193 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:53:55.733909 19198 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:53:55.733882 19201 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:53:55.733958 19199 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:53:55.734207 19193 server_base.cc:1034] running on GCE node
I20260806 22:53:55.734406 19193 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:53:55.734639 19193 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:53:55.735780 19193 hybrid_clock.cc:648] HybridClock initialized: now 1786056835735763 us; error 33 us; skew 500 ppm
Aug 06 22:53:55 dist-test-slave-1xrh krb5kdc[19180](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056835, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:53:55.738476 19193 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 22:53:55.738955 19193 webserver.cc:460] Webserver started at http://127.0.0.1:37773/ using document root <none> and password file <none>
I20260806 22:53:55.739240 19193 fs_manager.cc:359] Metadata directory not provided
I20260806 22:53:55.739310 19193 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:53:55.739444 19193 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:53:55.740437 19193 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786056586788339-29547-0/minicluster-data/master-0/data/instance:
uuid: "b73e395c66e844ad87d33cdb6518b5b4"
format_stamp: "Formatted at 2026-08-06 22:53:55 on dist-test-slave-1xrh"
server_key: "f731a99c073fd706ce5fce678182c9a0"
server_key_iv: "237280247947452c310fdda129dffdd1"
server_key_version: "encryptionkey@0"
I20260806 22:53:55.740803 19193 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786056586788339-29547-0/minicluster-data/master-0/wal/instance:
uuid: "b73e395c66e844ad87d33cdb6518b5b4"
format_stamp: "Formatted at 2026-08-06 22:53:55 on dist-test-slave-1xrh"
server_key: "f731a99c073fd706ce5fce678182c9a0"
server_key_iv: "237280247947452c310fdda129dffdd1"
server_key_version: "encryptionkey@0"
I20260806 22:53:55.742102 19193 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:53:55.742822 19208 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:53:55.742993 19193 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:53:55.743093 19193 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786056586788339-29547-0/minicluster-data/master-0/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786056586788339-29547-0/minicluster-data/master-0/wal
uuid: "b73e395c66e844ad87d33cdb6518b5b4"
format_stamp: "Formatted at 2026-08-06 22:53:55 on dist-test-slave-1xrh"
server_key: "f731a99c073fd706ce5fce678182c9a0"
server_key_iv: "237280247947452c310fdda129dffdd1"
server_key_version: "encryptionkey@0"
I20260806 22:53:55.743201 19193 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786056586788339-29547-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786056586788339-29547-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786056586788339-29547-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:53:55.767246 19193 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:53:55.768950 19193 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:53:55.769106 19193 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:53:55.773136 19193 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37319
I20260806 22:53:55.773172 19263 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37319 every 8 connection(s)
I20260806 22:53:55.773501 19193 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
I20260806 22:53:55.774339 19264 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:53:55.776572 19264 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b73e395c66e844ad87d33cdb6518b5b4: Bootstrap starting.
I20260806 22:53:55.777170 19264 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b73e395c66e844ad87d33cdb6518b5b4: Neither blocks nor log segments found. Creating new log.
I20260806 22:53:55.777535 19264 log.cc:824] T 00000000000000000000000000000000 P b73e395c66e844ad87d33cdb6518b5b4: Log is configured to *not* fsync() on all Append() calls
I20260806 22:53:55.778234 19264 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b73e395c66e844ad87d33cdb6518b5b4: No bootstrap required, opened a new log
I20260806 22:53:55.779831 19264 raft_consensus.cc:348] T 00000000000000000000000000000000 P b73e395c66e844ad87d33cdb6518b5b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b73e395c66e844ad87d33cdb6518b5b4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37319 } }
I20260806 22:53:55.779966 19264 raft_consensus.cc:374] T 00000000000000000000000000000000 P b73e395c66e844ad87d33cdb6518b5b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:53:55.780033 19264 raft_consensus.cc:729] T 00000000000000000000000000000000 P b73e395c66e844ad87d33cdb6518b5b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b73e395c66e844ad87d33cdb6518b5b4, State: Initialized, Role: FOLLOWER
I20260806 22:53:55.780189 19264 consensus_queue.cc:260] T 00000000000000000000000000000000 P b73e395c66e844ad87d33cdb6518b5b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b73e395c66e844ad87d33cdb6518b5b4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37319 } }
I20260806 22:53:55.780273 19264 raft_consensus.cc:388] T 00000000000000000000000000000000 P b73e395c66e844ad87d33cdb6518b5b4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:53:55.780311 19264 raft_consensus.cc:482] T 00000000000000000000000000000000 P b73e395c66e844ad87d33cdb6518b5b4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:53:55.780364 19264 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b73e395c66e844ad87d33cdb6518b5b4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:53:55.780362 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 19193
I20260806 22:53:55.780490 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786056586788339-29547-0/minicluster-data/master-0/wal/instance
I20260806 22:53:55.780702 29547 external_mini_cluster.cc:1428] Setting key dd1b83b62d15fd2ce475e44daba8e38a
I20260806 22:53:55.781038 19264 raft_consensus.cc:504] T 00000000000000000000000000000000 P b73e395c66e844ad87d33cdb6518b5b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b73e395c66e844ad87d33cdb6518b5b4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37319 } }
I20260806 22:53:55.781177 19264 leader_election.cc:302] T 00000000000000000000000000000000 P b73e395c66e844ad87d33cdb6518b5b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b73e395c66e844ad87d33cdb6518b5b4; no voters:
I20260806 22:53:55.781351 19264 leader_election.cc:288] T 00000000000000000000000000000000 P b73e395c66e844ad87d33cdb6518b5b4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:53:55.781392 19269 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b73e395c66e844ad87d33cdb6518b5b4 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:53:55.781560 19269 raft_consensus.cc:686] T 00000000000000000000000000000000 P b73e395c66e844ad87d33cdb6518b5b4 [term 1 LEADER]: Becoming Leader. State: Replica: b73e395c66e844ad87d33cdb6518b5b4, State: Running, Role: LEADER
I20260806 22:53:55.781603 19264 sys_catalog.cc:564] T 00000000000000000000000000000000 P b73e395c66e844ad87d33cdb6518b5b4 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:53:55.781726 19269 consensus_queue.cc:237] T 00000000000000000000000000000000 P b73e395c66e844ad87d33cdb6518b5b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b73e395c66e844ad87d33cdb6518b5b4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37319 } }
I20260806 22:53:55.782225 19270 sys_catalog.cc:455] T 00000000000000000000000000000000 P b73e395c66e844ad87d33cdb6518b5b4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b73e395c66e844ad87d33cdb6518b5b4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b73e395c66e844ad87d33cdb6518b5b4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37319 } } }
I20260806 22:53:55.782327 19270 sys_catalog.cc:458] T 00000000000000000000000000000000 P b73e395c66e844ad87d33cdb6518b5b4 [sys.catalog]: This master's current role is: LEADER
I20260806 22:53:55.782224 19271 sys_catalog.cc:455] T 00000000000000000000000000000000 P b73e395c66e844ad87d33cdb6518b5b4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b73e395c66e844ad87d33cdb6518b5b4. Latest consensus state: current_term: 1 leader_uuid: "b73e395c66e844ad87d33cdb6518b5b4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b73e395c66e844ad87d33cdb6518b5b4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37319 } } }
I20260806 22:53:55.782481 19271 sys_catalog.cc:458] T 00000000000000000000000000000000 P b73e395c66e844ad87d33cdb6518b5b4 [sys.catalog]: This master's current role is: LEADER
I20260806 22:53:55.782912 19278 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:53:55.783478 19278 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:53:55.785096 19278 catalog_manager.cc:1347] Generated new cluster ID: 47f30f2ccacc44fda2d6059fb1622352
I20260806 22:53:55.785148 19278 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 22:53:55 dist-test-slave-1xrh krb5kdc[19180](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056835, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 22:53:55.791280 19268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:55.781174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.186:48915 (local address 127.0.0.1:37319)
0806 22:53:55.781381 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0806 22:53:55.781391 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:53:55.781440 (+ 49us) server_negotiation.cc:407] Connection header received
0806 22:53:55.781569 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:53:55.781600 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:53:55.781661 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:53:55.781804 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:53:55.782857 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:55.784298 (+ 1441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:55.785109 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:55.785557 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:55.785688 (+ 131us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:55.788032 (+ 2344us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:53:55.788035 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:53:55.788036 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:53:55.788046 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:53:55.789703 (+ 1657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:53:55.790112 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:53:55.790114 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:53:55.790115 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:53:55.790214 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:53:55.790447 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:53:55.790448 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:53:55.790449 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:53:55.790671 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:53:55.790843 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0806 22:53:55.790965 (+ 122us) server_negotiation.cc:299] Negotiation successful
0806 22:53:55.791081 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":73,"threads_started":1}
I20260806 22:53:55.792232 29547 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:53:55.793726 19278 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:53:55.794426 19278 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:53:55.799682 19278 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b73e395c66e844ad87d33cdb6518b5b4: Generated new TSK 0
I20260806 22:53:55.799784 19268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:55.793550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.186:60903 (local address 127.0.0.1:37319)
0806 22:53:55.793633 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0806 22:53:55.793634 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:53:55.793685 (+ 51us) server_negotiation.cc:407] Connection header received
0806 22:53:55.793761 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:53:55.793762 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:53:55.793771 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:53:55.793816 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:53:55.794329 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:55.794870 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:55.795658 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:55.795896 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:55.795978 (+ 82us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:55.797090 (+ 1112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:53:55.797091 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:53:55.797092 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:53:55.797098 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:53:55.798461 (+ 1363us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:53:55.798851 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:53:55.798852 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:53:55.798852 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:53:55.798909 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:53:55.799148 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:53:55.799149 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:53:55.799149 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:53:55.799256 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:53:55.799520 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0806 22:53:55.799572 (+ 52us) server_negotiation.cc:299] Negotiation successful
0806 22:53:55.799645 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 22:53:55.799846 19278 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:53:55.801664 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 19193
W20260806 22:53:55.807430 19177 connection.cc:535] client connection to 127.0.0.1:37319 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37319): Connection reset by peer (error 104)
2026-08-06T22:53:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1323 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786056586788339-29547-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:53:55 dist-test-slave-1xrh krb5kdc[19295](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:53:55 dist-test-slave-1xrh krb5kdc[19295](info): set up 2 sockets
Aug 06 22:53:55 dist-test-slave-1xrh krb5kdc[19295](info): commencing operation
krb5kdc: starting...
W20260806 22:53:56.854297 29547 mini_kdc.cc:121] Time spent starting KDC: real 1.033s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786056586788339-29547-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786056586788339-29547-0/krb5kdc/test-user.keytab.
Aug 06 22:53:56 dist-test-slave-1xrh krb5kdc[19295](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056836, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:53:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:53:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786056586788339-29547-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786056586788339-29547-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 22:53:56.970810 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786056586788339-29547-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786056586788339-29547-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786056586788339-29547-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36107
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786056586788339-29547-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36107
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:53:57.050246 19308 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:53:57.050432 19308 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:53:57.050454 19308 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:53:57.051518 19308 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:53:57.051550 19308 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:53:57.051564 19308 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:53:57.051576 19308 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:53:57.051587 19308 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:53:57.052940 19308 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786056586788339-29547-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786056586788339-29547-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36107
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786056586788339-29547-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36107
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19308
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786056586788339-29547-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:53:57.053123 19308 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:53:57.053282 19308 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:53:57.057926 19314 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:53:57.057933 19317 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:53:57.057957 19313 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:53:57.058192 19308 server_base.cc:1034] running on GCE node
I20260806 22:53:57.058393 19308 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:53:57.058607 19308 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:53:57.059738 19308 hybrid_clock.cc:648] HybridClock initialized: now 1786056837059727 us; error 26 us; skew 500 ppm
Aug 06 22:53:57 dist-test-slave-1xrh krb5kdc[19295](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056837, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:53:57.062129 19308 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 22:53:57.062572 19308 webserver.cc:460] Webserver started at http://127.0.0.1:37449/ using document root <none> and password file <none>
I20260806 22:53:57.062784 19308 fs_manager.cc:359] Metadata directory not provided
I20260806 22:53:57.062849 19308 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:53:57.062996 19308 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:53:57.064076 19308 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786056586788339-29547-0/minicluster-data/master-0/data/instance:
uuid: "d2bb473780d7412ea5ee6c1ded864b1d"
format_stamp: "Formatted at 2026-08-06 22:53:57 on dist-test-slave-1xrh"
server_key: "4fd20d6f94082af6af768e895fa99ed5"
server_key_iv: "39ecc6822e109ab006a3daf4df76ff0c"
server_key_version: "encryptionkey@0"
I20260806 22:53:57.064429 19308 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786056586788339-29547-0/minicluster-data/master-0/wal/instance:
uuid: "d2bb473780d7412ea5ee6c1ded864b1d"
format_stamp: "Formatted at 2026-08-06 22:53:57 on dist-test-slave-1xrh"
server_key: "4fd20d6f94082af6af768e895fa99ed5"
server_key_iv: "39ecc6822e109ab006a3daf4df76ff0c"
server_key_version: "encryptionkey@0"
I20260806 22:53:57.065730 19308 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:53:57.066378 19325 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:53:57.066547 19308 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:53:57.066641 19308 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786056586788339-29547-0/minicluster-data/master-0/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786056586788339-29547-0/minicluster-data/master-0/wal
uuid: "d2bb473780d7412ea5ee6c1ded864b1d"
format_stamp: "Formatted at 2026-08-06 22:53:57 on dist-test-slave-1xrh"
server_key: "4fd20d6f94082af6af768e895fa99ed5"
server_key_iv: "39ecc6822e109ab006a3daf4df76ff0c"
server_key_version: "encryptionkey@0"
I20260806 22:53:57.066725 19308 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786056586788339-29547-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786056586788339-29547-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786056586788339-29547-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:53:57.110056 19308 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:53:57.112385 19308 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:53:57.112550 19308 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:53:57.116386 19308 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36107
I20260806 22:53:57.116425 19379 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36107 every 8 connection(s)
I20260806 22:53:57.116809 19308 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
I20260806 22:53:57.117660 19380 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:53:57.119758 19380 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d2bb473780d7412ea5ee6c1ded864b1d: Bootstrap starting.
I20260806 22:53:57.120308 19380 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d2bb473780d7412ea5ee6c1ded864b1d: Neither blocks nor log segments found. Creating new log.
I20260806 22:53:57.120584 19380 log.cc:824] T 00000000000000000000000000000000 P d2bb473780d7412ea5ee6c1ded864b1d: Log is configured to *not* fsync() on all Append() calls
I20260806 22:53:57.121245 19380 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d2bb473780d7412ea5ee6c1ded864b1d: No bootstrap required, opened a new log
I20260806 22:53:57.122794 19380 raft_consensus.cc:348] T 00000000000000000000000000000000 P d2bb473780d7412ea5ee6c1ded864b1d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2bb473780d7412ea5ee6c1ded864b1d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36107 } }
I20260806 22:53:57.122946 19380 raft_consensus.cc:374] T 00000000000000000000000000000000 P d2bb473780d7412ea5ee6c1ded864b1d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:53:57.123023 19380 raft_consensus.cc:729] T 00000000000000000000000000000000 P d2bb473780d7412ea5ee6c1ded864b1d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2bb473780d7412ea5ee6c1ded864b1d, State: Initialized, Role: FOLLOWER
I20260806 22:53:57.123214 19380 consensus_queue.cc:260] T 00000000000000000000000000000000 P d2bb473780d7412ea5ee6c1ded864b1d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2bb473780d7412ea5ee6c1ded864b1d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36107 } }
I20260806 22:53:57.123301 19380 raft_consensus.cc:388] T 00000000000000000000000000000000 P d2bb473780d7412ea5ee6c1ded864b1d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:53:57.123346 19380 raft_consensus.cc:482] T 00000000000000000000000000000000 P d2bb473780d7412ea5ee6c1ded864b1d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:53:57.123394 19380 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d2bb473780d7412ea5ee6c1ded864b1d [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:53:57.123896 19380 raft_consensus.cc:504] T 00000000000000000000000000000000 P d2bb473780d7412ea5ee6c1ded864b1d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2bb473780d7412ea5ee6c1ded864b1d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36107 } }
I20260806 22:53:57.124019 19380 leader_election.cc:302] T 00000000000000000000000000000000 P d2bb473780d7412ea5ee6c1ded864b1d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d2bb473780d7412ea5ee6c1ded864b1d; no voters:
I20260806 22:53:57.124188 19380 leader_election.cc:288] T 00000000000000000000000000000000 P d2bb473780d7412ea5ee6c1ded864b1d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:53:57.124219 19383 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d2bb473780d7412ea5ee6c1ded864b1d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:53:57.124397 19383 raft_consensus.cc:686] T 00000000000000000000000000000000 P d2bb473780d7412ea5ee6c1ded864b1d [term 1 LEADER]: Becoming Leader. State: Replica: d2bb473780d7412ea5ee6c1ded864b1d, State: Running, Role: LEADER
I20260806 22:53:57.124428 19380 sys_catalog.cc:564] T 00000000000000000000000000000000 P d2bb473780d7412ea5ee6c1ded864b1d [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:53:57.124574 19383 consensus_queue.cc:237] T 00000000000000000000000000000000 P d2bb473780d7412ea5ee6c1ded864b1d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2bb473780d7412ea5ee6c1ded864b1d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36107 } }
I20260806 22:53:57.124955 19384 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2bb473780d7412ea5ee6c1ded864b1d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d2bb473780d7412ea5ee6c1ded864b1d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2bb473780d7412ea5ee6c1ded864b1d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36107 } } }
I20260806 22:53:57.124979 19385 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2bb473780d7412ea5ee6c1ded864b1d [sys.catalog]: SysCatalogTable state changed. Reason: New leader d2bb473780d7412ea5ee6c1ded864b1d. Latest consensus state: current_term: 1 leader_uuid: "d2bb473780d7412ea5ee6c1ded864b1d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2bb473780d7412ea5ee6c1ded864b1d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36107 } } }
I20260806 22:53:57.125118 19384 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2bb473780d7412ea5ee6c1ded864b1d [sys.catalog]: This master's current role is: LEADER
I20260806 22:53:57.125136 19385 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2bb473780d7412ea5ee6c1ded864b1d [sys.catalog]: This master's current role is: LEADER
I20260806 22:53:57.125560 19388 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:53:57.125694 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 19308
I20260806 22:53:57.125797 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786056586788339-29547-0/minicluster-data/master-0/wal/instance
I20260806 22:53:57.125999 29547 external_mini_cluster.cc:1428] Setting key 65f82745be2200dc855ca4a37583b4ff
I20260806 22:53:57.126072 19388 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:53:57.131518 19388 catalog_manager.cc:1347] Generated new cluster ID: f460565fd2a6457387c1357e2d5f38a4
I20260806 22:53:57.131572 19388 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:53:57.131639 19403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:57.126592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.186:36637 (local address 127.0.0.1:36107)
0806 22:53:57.126770 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0806 22:53:57.126779 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:53:57.126816 (+ 37us) server_negotiation.cc:407] Connection header received
0806 22:53:57.126925 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:53:57.126932 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:53:57.126987 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:53:57.127151 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:53:57.127383 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:57.128714 (+ 1331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:57.129496 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:57.130118 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:57.130231 (+ 113us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:57.130824 (+ 593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:53:57.130828 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:53:57.130828 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:53:57.131028 (+ 200us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:53:57.131065 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:53:57.131078 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:53:57.131217 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0806 22:53:57.131292 (+ 75us) server_negotiation.cc:299] Negotiation successful
0806 22:53:57.131419 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":75,"threads_started":1}
I20260806 22:53:57.132642 29547 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:53:57.137467 19403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:57.134078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.186:47159 (local address 127.0.0.1:36107)
0806 22:53:57.134123 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 22:53:57.134124 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:53:57.134279 (+ 155us) server_negotiation.cc:407] Connection header received
0806 22:53:57.134339 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:53:57.134340 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:53:57.134349 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:53:57.134405 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:53:57.134541 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:57.135243 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:57.136254 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:57.136556 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:57.136611 (+ 55us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:57.137239 (+ 628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:53:57.137240 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:53:57.137240 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:53:57.137244 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:53:57.137255 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:53:57.137259 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:53:57.137305 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0806 22:53:57.137391 (+ 86us) server_negotiation.cc:299] Negotiation successful
0806 22:53:57.137402 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 22:53:57.147812 19388 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:53:57.148260 19388 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:53:57.153519 19388 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d2bb473780d7412ea5ee6c1ded864b1d: Generated new TSK 0
I20260806 22:53:57.153689 19388 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:53:57.156172 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 19308
W20260806 22:53:57.163638 19293 connection.cc:535] client connection to 127.0.0.1:36107 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36107): Connection reset by peer (error 104)
2026-08-06T22:53:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1356 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786056586788339-29547-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:53:57 dist-test-slave-1xrh krb5kdc[19412](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:53:57 dist-test-slave-1xrh krb5kdc[19412](info): set up 2 sockets
Aug 06 22:53:57 dist-test-slave-1xrh krb5kdc[19412](info): commencing operation
krb5kdc: starting...
W20260806 22:53:58.202775 29547 mini_kdc.cc:121] Time spent starting KDC: real 1.025s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786056586788339-29547-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786056586788339-29547-0/krb5kdc/test-user.keytab.
Aug 06 22:53:58 dist-test-slave-1xrh krb5kdc[19412](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056838, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:53:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:53:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786056586788339-29547-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786056586788339-29547-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 22:53:58.322340 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786056586788339-29547-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786056586788339-29547-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786056586788339-29547-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41877
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786056586788339-29547-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41877
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:53:58.404659 19425 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:53:58.404886 19425 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:53:58.404923 19425 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:53:58.406020 19425 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:53:58.406088 19425 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:53:58.406111 19425 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:53:58.406132 19425 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:53:58.406160 19425 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:53:58.407585 19425 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786056586788339-29547-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786056586788339-29547-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41877
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786056586788339-29547-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41877
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19425
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786056586788339-29547-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:53:58.407866 19425 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:53:58.408092 19425 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:53:58.412814 19431 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:53:58.412829 19433 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:53:58.413055 19425 server_base.cc:1034] running on GCE node
W20260806 22:53:58.412896 19430 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:53:58.413357 19425 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:53:58.413635 19425 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:53:58.414795 19425 hybrid_clock.cc:648] HybridClock initialized: now 1786056838414786 us; error 26 us; skew 500 ppm
Aug 06 22:53:58 dist-test-slave-1xrh krb5kdc[19412](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056838, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:53:58.421983 19425 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 22:53:58.422436 19425 webserver.cc:460] Webserver started at http://127.0.0.1:35591/ using document root <none> and password file <none>
I20260806 22:53:58.422686 19425 fs_manager.cc:359] Metadata directory not provided
I20260806 22:53:58.422755 19425 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:53:58.422912 19425 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:53:58.424080 19425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786056586788339-29547-0/minicluster-data/master-0/data/instance:
uuid: "821034393d774d66be5e072d5537f841"
format_stamp: "Formatted at 2026-08-06 22:53:58 on dist-test-slave-1xrh"
server_key: "afa001de434c0dfe19af64ac172dd6ae"
server_key_iv: "6e5ecc39475a53b22a8b2960b59438a5"
server_key_version: "encryptionkey@0"
I20260806 22:53:58.424441 19425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786056586788339-29547-0/minicluster-data/master-0/wal/instance:
uuid: "821034393d774d66be5e072d5537f841"
format_stamp: "Formatted at 2026-08-06 22:53:58 on dist-test-slave-1xrh"
server_key: "afa001de434c0dfe19af64ac172dd6ae"
server_key_iv: "6e5ecc39475a53b22a8b2960b59438a5"
server_key_version: "encryptionkey@0"
I20260806 22:53:58.425760 19425 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:53:58.426492 19440 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:53:58.426658 19425 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:53:58.426748 19425 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786056586788339-29547-0/minicluster-data/master-0/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786056586788339-29547-0/minicluster-data/master-0/wal
uuid: "821034393d774d66be5e072d5537f841"
format_stamp: "Formatted at 2026-08-06 22:53:58 on dist-test-slave-1xrh"
server_key: "afa001de434c0dfe19af64ac172dd6ae"
server_key_iv: "6e5ecc39475a53b22a8b2960b59438a5"
server_key_version: "encryptionkey@0"
I20260806 22:53:58.426833 19425 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786056586788339-29547-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786056586788339-29547-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786056586788339-29547-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:53:58.430078 19425 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:53:58.431953 19425 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:53:58.432113 19425 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:53:58.436081 19425 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41877
I20260806 22:53:58.436111 19493 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41877 every 8 connection(s)
I20260806 22:53:58.436416 19425 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
I20260806 22:53:58.436851 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 19425
I20260806 22:53:58.436995 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786056586788339-29547-0/minicluster-data/master-0/wal/instance
I20260806 22:53:58.437179 29547 external_mini_cluster.cc:1428] Setting key 858a2bf4696627d433854e863d07fc84
I20260806 22:53:58.437614 19494 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:53:58.439719 19497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:58.437677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.186:38145 (local address 127.0.0.1:41877)
0806 22:53:58.437882 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0806 22:53:58.437891 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:53:58.437925 (+ 34us) server_negotiation.cc:407] Connection header received
0806 22:53:58.438425 (+ 500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:53:58.438433 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:53:58.438465 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:53:58.438614 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:53:58.439262 (+ 648us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:53:58.439265 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:53:58.439265 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:53:58.439284 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:53:58.439319 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:53:58.439329 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:53:58.439377 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 22:53:58.439478 (+ 101us) server_negotiation.cc:299] Negotiation successful
0806 22:53:58.439510 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":88,"threads_started":1}
I20260806 22:53:58.440663 29547 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:53:58.440963 19494 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 821034393d774d66be5e072d5537f841: Bootstrap starting.
I20260806 22:53:58.441566 19494 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 821034393d774d66be5e072d5537f841: Neither blocks nor log segments found. Creating new log.
I20260806 22:53:58.441934 19494 log.cc:824] T 00000000000000000000000000000000 P 821034393d774d66be5e072d5537f841: Log is configured to *not* fsync() on all Append() calls
I20260806 22:53:58.442664 19494 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 821034393d774d66be5e072d5537f841: No bootstrap required, opened a new log
I20260806 22:53:58.443048 19497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:58.441984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.186:45123 (local address 127.0.0.1:41877)
0806 22:53:58.442023 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0806 22:53:58.442025 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:53:58.442142 (+ 117us) server_negotiation.cc:407] Connection header received
0806 22:53:58.442216 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:53:58.442217 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:53:58.442221 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:53:58.442274 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:53:58.442845 (+ 571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:53:58.442846 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:53:58.442846 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:53:58.442850 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:53:58.442861 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:53:58.442865 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:53:58.442911 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0806 22:53:58.442965 (+ 54us) server_negotiation.cc:299] Negotiation successful
0806 22:53:58.442979 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260806 22:53:58.444665 19494 raft_consensus.cc:348] T 00000000000000000000000000000000 P 821034393d774d66be5e072d5537f841 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "821034393d774d66be5e072d5537f841" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41877 } }
I20260806 22:53:58.444790 19494 raft_consensus.cc:374] T 00000000000000000000000000000000 P 821034393d774d66be5e072d5537f841 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:53:58.444813 19494 raft_consensus.cc:729] T 00000000000000000000000000000000 P 821034393d774d66be5e072d5537f841 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 821034393d774d66be5e072d5537f841, State: Initialized, Role: FOLLOWER
I20260806 22:53:58.444975 19494 consensus_queue.cc:260] T 00000000000000000000000000000000 P 821034393d774d66be5e072d5537f841 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "821034393d774d66be5e072d5537f841" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41877 } }
I20260806 22:53:58.445048 19494 raft_consensus.cc:388] T 00000000000000000000000000000000 P 821034393d774d66be5e072d5537f841 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:53:58.445092 19494 raft_consensus.cc:482] T 00000000000000000000000000000000 P 821034393d774d66be5e072d5537f841 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:53:58.445143 19494 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 821034393d774d66be5e072d5537f841 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:53:58.445694 19494 raft_consensus.cc:504] T 00000000000000000000000000000000 P 821034393d774d66be5e072d5537f841 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "821034393d774d66be5e072d5537f841" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41877 } }
I20260806 22:53:58.445801 19494 leader_election.cc:302] T 00000000000000000000000000000000 P 821034393d774d66be5e072d5537f841 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 821034393d774d66be5e072d5537f841; no voters:
I20260806 22:53:58.445957 19494 leader_election.cc:288] T 00000000000000000000000000000000 P 821034393d774d66be5e072d5537f841 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:53:58.446012 19504 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 821034393d774d66be5e072d5537f841 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:53:58.446195 19504 raft_consensus.cc:686] T 00000000000000000000000000000000 P 821034393d774d66be5e072d5537f841 [term 1 LEADER]: Becoming Leader. State: Replica: 821034393d774d66be5e072d5537f841, State: Running, Role: LEADER
I20260806 22:53:58.446259 19494 sys_catalog.cc:564] T 00000000000000000000000000000000 P 821034393d774d66be5e072d5537f841 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:53:58.446396 19504 consensus_queue.cc:237] T 00000000000000000000000000000000 P 821034393d774d66be5e072d5537f841 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "821034393d774d66be5e072d5537f841" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41877 } }
I20260806 22:53:58.446767 19505 sys_catalog.cc:455] T 00000000000000000000000000000000 P 821034393d774d66be5e072d5537f841 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "821034393d774d66be5e072d5537f841" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "821034393d774d66be5e072d5537f841" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41877 } } }
I20260806 22:53:58.446772 19506 sys_catalog.cc:455] T 00000000000000000000000000000000 P 821034393d774d66be5e072d5537f841 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 821034393d774d66be5e072d5537f841. Latest consensus state: current_term: 1 leader_uuid: "821034393d774d66be5e072d5537f841" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "821034393d774d66be5e072d5537f841" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41877 } } }
I20260806 22:53:58.446975 19506 sys_catalog.cc:458] T 00000000000000000000000000000000 P 821034393d774d66be5e072d5537f841 [sys.catalog]: This master's current role is: LEADER
I20260806 22:53:58.446913 19505 sys_catalog.cc:458] T 00000000000000000000000000000000 P 821034393d774d66be5e072d5537f841 [sys.catalog]: This master's current role is: LEADER
I20260806 22:53:58.447372 19513 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:53:58.447842 19513 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:53:58.449262 19513 catalog_manager.cc:1347] Generated new cluster ID: c61e1716cba3451bb2dd45e87738ad59
I20260806 22:53:58.449309 19513 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:53:58.454602 19513 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:53:58.454768 19513 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:53:58.459928 19513 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 821034393d774d66be5e072d5537f841: Generated new TSK 0
I20260806 22:53:58.460078 19513 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:53:58.464610 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 19425
2026-08-06T22:53:58Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1306 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786056586788339-29547-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:53:58 dist-test-slave-1xrh krb5kdc[19525](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:53:58 dist-test-slave-1xrh krb5kdc[19525](info): set up 2 sockets
Aug 06 22:53:58 dist-test-slave-1xrh krb5kdc[19525](info): commencing operation
krb5kdc: starting...
W20260806 22:53:59.526901 29547 mini_kdc.cc:121] Time spent starting KDC: real 1.043s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786056586788339-29547-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786056586788339-29547-0/krb5kdc/test-user.keytab.
Aug 06 22:53:59 dist-test-slave-1xrh krb5kdc[19525](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056839, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:53:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:53:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786056586788339-29547-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786056586788339-29547-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 22:53:59.645663 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786056586788339-29547-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786056586788339-29547-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786056586788339-29547-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34573
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786056586788339-29547-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34573
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:53:59.725891 19540 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:53:59.726090 19540 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:53:59.726114 19540 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:53:59.727205 19540 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:53:59.727238 19540 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:53:59.727252 19540 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:53:59.727264 19540 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:53:59.727276 19540 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:53:59.728647 19540 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786056586788339-29547-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786056586788339-29547-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34573
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786056586788339-29547-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34573
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19540
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786056586788339-29547-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:53:59.728830 19540 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:53:59.728986 19540 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:53:59.733762 19545 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:53:59.733982 19540 server_base.cc:1034] running on GCE node
W20260806 22:53:59.733752 19548 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:53:59.733788 19546 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:53:59.734385 19540 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:53:59.734639 19540 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:53:59.735772 19540 hybrid_clock.cc:648] HybridClock initialized: now 1786056839735752 us; error 37 us; skew 500 ppm
Aug 06 22:53:59 dist-test-slave-1xrh krb5kdc[19525](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056839, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:53:59.738090 19540 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 22:53:59.738498 19540 webserver.cc:460] Webserver started at http://127.0.0.1:34703/ using document root <none> and password file <none>
I20260806 22:53:59.738739 19540 fs_manager.cc:359] Metadata directory not provided
I20260806 22:53:59.738785 19540 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:53:59.738926 19540 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:53:59.740033 19540 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786056586788339-29547-0/minicluster-data/master-0/data/instance:
uuid: "4e7a6206802a4ca9bb7c7359a5dab55d"
format_stamp: "Formatted at 2026-08-06 22:53:59 on dist-test-slave-1xrh"
server_key: "8ccdc88d1a62404bc34e6358161efc58"
server_key_iv: "1f7e8746931fe88d25bdc46cdfa26f28"
server_key_version: "encryptionkey@0"
I20260806 22:53:59.740391 19540 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786056586788339-29547-0/minicluster-data/master-0/wal/instance:
uuid: "4e7a6206802a4ca9bb7c7359a5dab55d"
format_stamp: "Formatted at 2026-08-06 22:53:59 on dist-test-slave-1xrh"
server_key: "8ccdc88d1a62404bc34e6358161efc58"
server_key_iv: "1f7e8746931fe88d25bdc46cdfa26f28"
server_key_version: "encryptionkey@0"
I20260806 22:53:59.741721 19540 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:53:59.742432 19556 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:53:59.742614 19540 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:53:59.742702 19540 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786056586788339-29547-0/minicluster-data/master-0/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786056586788339-29547-0/minicluster-data/master-0/wal
uuid: "4e7a6206802a4ca9bb7c7359a5dab55d"
format_stamp: "Formatted at 2026-08-06 22:53:59 on dist-test-slave-1xrh"
server_key: "8ccdc88d1a62404bc34e6358161efc58"
server_key_iv: "1f7e8746931fe88d25bdc46cdfa26f28"
server_key_version: "encryptionkey@0"
I20260806 22:53:59.742754 19540 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786056586788339-29547-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786056586788339-29547-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786056586788339-29547-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:53:59.766458 19540 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:53:59.768774 19540 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:53:59.768886 19540 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:53:59.773013 19540 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34573
I20260806 22:53:59.773092 19610 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34573 every 8 connection(s)
I20260806 22:53:59.773415 19540 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
I20260806 22:53:59.774277 19612 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:53:59.776609 19612 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e7a6206802a4ca9bb7c7359a5dab55d: Bootstrap starting.
I20260806 22:53:59.777212 19612 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4e7a6206802a4ca9bb7c7359a5dab55d: Neither blocks nor log segments found. Creating new log.
I20260806 22:53:59.777516 19612 log.cc:824] T 00000000000000000000000000000000 P 4e7a6206802a4ca9bb7c7359a5dab55d: Log is configured to *not* fsync() on all Append() calls
I20260806 22:53:59.778213 19612 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e7a6206802a4ca9bb7c7359a5dab55d: No bootstrap required, opened a new log
I20260806 22:53:59.779894 19612 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4e7a6206802a4ca9bb7c7359a5dab55d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e7a6206802a4ca9bb7c7359a5dab55d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34573 } }
I20260806 22:53:59.779995 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 19540
I20260806 22:53:59.780052 19612 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4e7a6206802a4ca9bb7c7359a5dab55d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:53:59.780112 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786056586788339-29547-0/minicluster-data/master-0/wal/instance
I20260806 22:53:59.780102 19612 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4e7a6206802a4ca9bb7c7359a5dab55d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e7a6206802a4ca9bb7c7359a5dab55d, State: Initialized, Role: FOLLOWER
I20260806 22:53:59.780315 29547 external_mini_cluster.cc:1428] Setting key a6e7e2a730486a61e96449723c34d672
I20260806 22:53:59.780273 19612 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4e7a6206802a4ca9bb7c7359a5dab55d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e7a6206802a4ca9bb7c7359a5dab55d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34573 } }
I20260806 22:53:59.780364 19612 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4e7a6206802a4ca9bb7c7359a5dab55d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:53:59.780428 19612 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4e7a6206802a4ca9bb7c7359a5dab55d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:53:59.780483 19612 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4e7a6206802a4ca9bb7c7359a5dab55d [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:53:59.781186 19612 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4e7a6206802a4ca9bb7c7359a5dab55d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e7a6206802a4ca9bb7c7359a5dab55d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34573 } }
I20260806 22:53:59.781316 19612 leader_election.cc:302] T 00000000000000000000000000000000 P 4e7a6206802a4ca9bb7c7359a5dab55d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4e7a6206802a4ca9bb7c7359a5dab55d; no voters:
I20260806 22:53:59.781487 19612 leader_election.cc:288] T 00000000000000000000000000000000 P 4e7a6206802a4ca9bb7c7359a5dab55d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:53:59.781522 19617 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4e7a6206802a4ca9bb7c7359a5dab55d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:53:59.781646 19617 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4e7a6206802a4ca9bb7c7359a5dab55d [term 1 LEADER]: Becoming Leader. State: Replica: 4e7a6206802a4ca9bb7c7359a5dab55d, State: Running, Role: LEADER
I20260806 22:53:59.781760 19612 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4e7a6206802a4ca9bb7c7359a5dab55d [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:53:59.781813 19617 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4e7a6206802a4ca9bb7c7359a5dab55d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e7a6206802a4ca9bb7c7359a5dab55d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34573 } }
I20260806 22:53:59.782296 19618 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e7a6206802a4ca9bb7c7359a5dab55d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4e7a6206802a4ca9bb7c7359a5dab55d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e7a6206802a4ca9bb7c7359a5dab55d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34573 } } }
I20260806 22:53:59.782380 19618 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e7a6206802a4ca9bb7c7359a5dab55d [sys.catalog]: This master's current role is: LEADER
I20260806 22:53:59.782600 19620 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e7a6206802a4ca9bb7c7359a5dab55d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4e7a6206802a4ca9bb7c7359a5dab55d. Latest consensus state: current_term: 1 leader_uuid: "4e7a6206802a4ca9bb7c7359a5dab55d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e7a6206802a4ca9bb7c7359a5dab55d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34573 } } }
I20260806 22:53:59.782737 19620 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e7a6206802a4ca9bb7c7359a5dab55d [sys.catalog]: This master's current role is: LEADER
W20260806 22:53:59.783615 19635 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4e7a6206802a4ca9bb7c7359a5dab55d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 22:53:59.783672 19635 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 22:53:59.783777 19627 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:53:59.784006 19627 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:53:59.785480 19627 catalog_manager.cc:1347] Generated new cluster ID: 6e5b2277fa22418f90e597c296ee8526
I20260806 22:53:59.785552 19627 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 22:53:59 dist-test-slave-1xrh krb5kdc[19525](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056839, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 22:53:59.793835 19616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:59.780815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.186:56073 (local address 127.0.0.1:34573)
0806 22:53:59.781429 (+ 614us) server_negotiation.cc:206] Beginning negotiation
0806 22:53:59.781439 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:53:59.781494 (+ 55us) server_negotiation.cc:407] Connection header received
0806 22:53:59.781628 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:53:59.781635 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:53:59.781699 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:53:59.782691 (+ 992us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:53:59.782802 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:59.784289 (+ 1487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:59.785229 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:59.786564 (+ 1335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:59.786674 (+ 110us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:59.789562 (+ 2888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:53:59.789566 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:53:59.789566 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:53:59.789578 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:53:59.792115 (+ 2537us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:53:59.792591 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:53:59.792593 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:53:59.792594 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:53:59.792704 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:53:59.792954 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:53:59.792955 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:53:59.792955 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:53:59.793237 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:53:59.793391 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0806 22:53:59.793534 (+ 143us) server_negotiation.cc:299] Negotiation successful
0806 22:53:59.793631 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":559,"thread_start_us":509,"threads_started":1}
I20260806 22:53:59.794888 29547 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:53:59.803490 19616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:53:59.796465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.186:56845 (local address 127.0.0.1:34573)
0806 22:53:59.796499 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0806 22:53:59.796501 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:53:59.796510 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:53:59.796540 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:53:59.796541 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:53:59.796550 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:53:59.796612 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:53:59.797173 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:59.797934 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:59.798880 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:53:59.799237 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:53:59.799307 (+ 70us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:53:59.800588 (+ 1281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:53:59.800589 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:53:59.800590 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:53:59.800598 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:53:59.802158 (+ 1560us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:53:59.802572 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:53:59.802573 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:53:59.802574 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:53:59.802631 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:53:59.802853 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:53:59.802854 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:53:59.802854 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:53:59.802979 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:53:59.803078 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 22:53:59.803233 (+ 155us) server_negotiation.cc:299] Negotiation successful
0806 22:53:59.803294 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 22:53:59.818399 19627 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:53:59.819092 19627 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:53:59.834470 19627 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4e7a6206802a4ca9bb7c7359a5dab55d: Generated new TSK 0
I20260806 22:53:59.834679 19627 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:53:59.844465 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 19540
W20260806 22:53:59.851173 19523 connection.cc:535] client connection to 127.0.0.1:34573 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34573): Connection reset by peer (error 104)
2026-08-06T22:53:59Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1380 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786056586788339-29547-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:53:59 dist-test-slave-1xrh krb5kdc[19645](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:53:59 dist-test-slave-1xrh krb5kdc[19645](info): set up 2 sockets
Aug 06 22:53:59 dist-test-slave-1xrh krb5kdc[19645](info): commencing operation
krb5kdc: starting...
W20260806 22:54:00.897069 29547 mini_kdc.cc:121] Time spent starting KDC: real 1.032s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786056586788339-29547-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786056586788339-29547-0/krb5kdc/test-user.keytab.
Aug 06 22:54:00 dist-test-slave-1xrh krb5kdc[19645](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056840, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:54:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:54:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786056586788339-29547-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786056586788339-29547-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 22:54:01.018188 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786056586788339-29547-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786056586788339-29547-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786056586788339-29547-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46725
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786056586788339-29547-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46725
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:54:01.098667 19658 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:54:01.098882 19658 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:54:01.098905 19658 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:54:01.100098 19658 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:54:01.100142 19658 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:54:01.100157 19658 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:54:01.100169 19658 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:54:01.100224 19658 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:54:01.101668 19658 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786056586788339-29547-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786056586788339-29547-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46725
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786056586788339-29547-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46725
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19658
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786056586788339-29547-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:54:01.101922 19658 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:54:01.102155 19658 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:54:01.107016 19667 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:54:01.107052 19663 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:54:01.106980 19664 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:54:01.107317 19658 server_base.cc:1034] running on GCE node
I20260806 22:54:01.107519 19658 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:54:01.107797 19658 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:54:01.108939 19658 hybrid_clock.cc:648] HybridClock initialized: now 1786056841108926 us; error 28 us; skew 500 ppm
Aug 06 22:54:01 dist-test-slave-1xrh krb5kdc[19645](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056841, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:54:01.111498 19658 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 22:54:01.111958 19658 webserver.cc:460] Webserver started at http://127.0.0.1:40907/ using document root <none> and password file <none>
I20260806 22:54:01.112185 19658 fs_manager.cc:359] Metadata directory not provided
I20260806 22:54:01.112251 19658 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:54:01.112416 19658 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:54:01.113561 19658 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786056586788339-29547-0/minicluster-data/master-0/data/instance:
uuid: "199b9e8fa7e74c58916d9293140ffdee"
format_stamp: "Formatted at 2026-08-06 22:54:01 on dist-test-slave-1xrh"
server_key: "eb31f7ffda123dc4ee79159124db95b5"
server_key_iv: "d9b8a1309420e4deb8a81c0c6ff4d2d4"
server_key_version: "encryptionkey@0"
I20260806 22:54:01.113931 19658 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786056586788339-29547-0/minicluster-data/master-0/wal/instance:
uuid: "199b9e8fa7e74c58916d9293140ffdee"
format_stamp: "Formatted at 2026-08-06 22:54:01 on dist-test-slave-1xrh"
server_key: "eb31f7ffda123dc4ee79159124db95b5"
server_key_iv: "d9b8a1309420e4deb8a81c0c6ff4d2d4"
server_key_version: "encryptionkey@0"
I20260806 22:54:01.115305 19658 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:54:01.116048 19675 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:54:01.116221 19658 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:54:01.116309 19658 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786056586788339-29547-0/minicluster-data/master-0/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786056586788339-29547-0/minicluster-data/master-0/wal
uuid: "199b9e8fa7e74c58916d9293140ffdee"
format_stamp: "Formatted at 2026-08-06 22:54:01 on dist-test-slave-1xrh"
server_key: "eb31f7ffda123dc4ee79159124db95b5"
server_key_iv: "d9b8a1309420e4deb8a81c0c6ff4d2d4"
server_key_version: "encryptionkey@0"
I20260806 22:54:01.116362 19658 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786056586788339-29547-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786056586788339-29547-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786056586788339-29547-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:54:01.132777 19658 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:54:01.134636 19658 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:54:01.134747 19658 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:54:01.138783 19658 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46725
I20260806 22:54:01.138839 19729 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46725 every 8 connection(s)
I20260806 22:54:01.139257 19658 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
I20260806 22:54:01.140180 19730 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:54:01.142449 19730 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 199b9e8fa7e74c58916d9293140ffdee: Bootstrap starting.
I20260806 22:54:01.142867 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 19658
I20260806 22:54:01.142987 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786056586788339-29547-0/minicluster-data/master-0/wal/instance
I20260806 22:54:01.143193 19730 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 199b9e8fa7e74c58916d9293140ffdee: Neither blocks nor log segments found. Creating new log.
I20260806 22:54:01.143230 29547 external_mini_cluster.cc:1428] Setting key c11bddd5f03817eec4533fbb0ef1bf9f
I20260806 22:54:01.143496 19730 log.cc:824] T 00000000000000000000000000000000 P 199b9e8fa7e74c58916d9293140ffdee: Log is configured to *not* fsync() on all Append() calls
I20260806 22:54:01.144295 19730 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 199b9e8fa7e74c58916d9293140ffdee: No bootstrap required, opened a new log
I20260806 22:54:01.146780 19730 raft_consensus.cc:348] T 00000000000000000000000000000000 P 199b9e8fa7e74c58916d9293140ffdee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "199b9e8fa7e74c58916d9293140ffdee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46725 } }
I20260806 22:54:01.146911 19730 raft_consensus.cc:374] T 00000000000000000000000000000000 P 199b9e8fa7e74c58916d9293140ffdee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:54:01.146934 19730 raft_consensus.cc:729] T 00000000000000000000000000000000 P 199b9e8fa7e74c58916d9293140ffdee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 199b9e8fa7e74c58916d9293140ffdee, State: Initialized, Role: FOLLOWER
I20260806 22:54:01.147058 19730 consensus_queue.cc:260] T 00000000000000000000000000000000 P 199b9e8fa7e74c58916d9293140ffdee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "199b9e8fa7e74c58916d9293140ffdee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46725 } }
I20260806 22:54:01.147177 19730 raft_consensus.cc:388] T 00000000000000000000000000000000 P 199b9e8fa7e74c58916d9293140ffdee [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:54:01.147217 19730 raft_consensus.cc:482] T 00000000000000000000000000000000 P 199b9e8fa7e74c58916d9293140ffdee [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:54:01.147271 19730 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 199b9e8fa7e74c58916d9293140ffdee [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:54:01.147945 19730 raft_consensus.cc:504] T 00000000000000000000000000000000 P 199b9e8fa7e74c58916d9293140ffdee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "199b9e8fa7e74c58916d9293140ffdee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46725 } }
I20260806 22:54:01.148052 19730 leader_election.cc:302] T 00000000000000000000000000000000 P 199b9e8fa7e74c58916d9293140ffdee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 199b9e8fa7e74c58916d9293140ffdee; no voters:
I20260806 22:54:01.148226 19730 leader_election.cc:288] T 00000000000000000000000000000000 P 199b9e8fa7e74c58916d9293140ffdee [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:54:01.148272 19735 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 199b9e8fa7e74c58916d9293140ffdee [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:54:01.148526 19735 raft_consensus.cc:686] T 00000000000000000000000000000000 P 199b9e8fa7e74c58916d9293140ffdee [term 1 LEADER]: Becoming Leader. State: Replica: 199b9e8fa7e74c58916d9293140ffdee, State: Running, Role: LEADER
I20260806 22:54:01.148546 19730 sys_catalog.cc:564] T 00000000000000000000000000000000 P 199b9e8fa7e74c58916d9293140ffdee [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:54:01.148712 19735 consensus_queue.cc:237] T 00000000000000000000000000000000 P 199b9e8fa7e74c58916d9293140ffdee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "199b9e8fa7e74c58916d9293140ffdee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46725 } }
I20260806 22:54:01.149858 19734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:54:01.143705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.186:47681 (local address 127.0.0.1:46725)
0806 22:54:01.143942 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0806 22:54:01.143951 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:54:01.143986 (+ 35us) server_negotiation.cc:407] Connection header received
0806 22:54:01.144573 (+ 587us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:54:01.144582 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:54:01.144644 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:54:01.144788 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:54:01.145046 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:54:01.146525 (+ 1479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:54:01.147553 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:54:01.148201 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:54:01.148312 (+ 111us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:54:01.149068 (+ 756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:54:01.149072 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:54:01.149073 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:54:01.149292 (+ 219us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:54:01.149336 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:54:01.149353 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:54:01.149446 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0806 22:54:01.149623 (+ 177us) server_negotiation.cc:299] Negotiation successful
0806 22:54:01.149654 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":118,"threads_started":1}
I20260806 22:54:01.149966 19737 sys_catalog.cc:455] T 00000000000000000000000000000000 P 199b9e8fa7e74c58916d9293140ffdee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "199b9e8fa7e74c58916d9293140ffdee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "199b9e8fa7e74c58916d9293140ffdee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46725 } } }
I20260806 22:54:01.150127 19737 sys_catalog.cc:458] T 00000000000000000000000000000000 P 199b9e8fa7e74c58916d9293140ffdee [sys.catalog]: This master's current role is: LEADER
W20260806 22:54:01.150780 19750 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 199b9e8fa7e74c58916d9293140ffdee: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260806 22:54:01.150875 29547 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:54:01.150156 19736 sys_catalog.cc:455] T 00000000000000000000000000000000 P 199b9e8fa7e74c58916d9293140ffdee [sys.catalog]: SysCatalogTable state changed. Reason: New leader 199b9e8fa7e74c58916d9293140ffdee. Latest consensus state: current_term: 1 leader_uuid: "199b9e8fa7e74c58916d9293140ffdee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "199b9e8fa7e74c58916d9293140ffdee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46725 } } }
W20260806 22:54:01.152166 19750 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 22:54:01.152212 19736 sys_catalog.cc:458] T 00000000000000000000000000000000 P 199b9e8fa7e74c58916d9293140ffdee [sys.catalog]: This master's current role is: LEADER
I20260806 22:54:01.152251 19751 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:54:01.152436 19751 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:54:01.154695 19751 catalog_manager.cc:1347] Generated new cluster ID: c12f646fbcf848f1b0f977c220e62b3e
I20260806 22:54:01.154752 19751 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:54:01.156323 19734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:54:01.152335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.186:34803 (local address 127.0.0.1:46725)
0806 22:54:01.152403 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0806 22:54:01.152405 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:54:01.153070 (+ 665us) server_negotiation.cc:407] Connection header received
0806 22:54:01.153145 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:54:01.153146 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:54:01.153157 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:54:01.153212 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:54:01.153346 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:54:01.154079 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:54:01.155011 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:54:01.155317 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:54:01.155403 (+ 86us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:54:01.156085 (+ 682us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:54:01.156086 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:54:01.156087 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:54:01.156091 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:54:01.156104 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:54:01.156108 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:54:01.156161 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 22:54:01.156220 (+ 59us) server_negotiation.cc:299] Negotiation successful
0806 22:54:01.156229 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 22:54:01.173054 19751 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:54:01.173830 19751 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:54:01.178658 19751 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 199b9e8fa7e74c58916d9293140ffdee: Generated new TSK 0
I20260806 22:54:01.178851 19751 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:54:01.184976 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 19658
W20260806 22:54:01.190812 19643 connection.cc:535] client connection to 127.0.0.1:46725 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46725): Connection reset by peer (error 104)
2026-08-06T22:54:01Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1339 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786056586788339-29547-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:54:01 dist-test-slave-1xrh krb5kdc[19761](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:54:01 dist-test-slave-1xrh krb5kdc[19761](info): set up 2 sockets
Aug 06 22:54:01 dist-test-slave-1xrh krb5kdc[19761](info): commencing operation
krb5kdc: starting...
W20260806 22:54:02.254638 29547 mini_kdc.cc:121] Time spent starting KDC: real 1.049s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786056586788339-29547-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786056586788339-29547-0/krb5kdc/test-user.keytab.
Aug 06 22:54:02 dist-test-slave-1xrh krb5kdc[19761](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056842, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:54:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:54:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786056586788339-29547-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786056586788339-29547-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 22:54:02.381754 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786056586788339-29547-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786056586788339-29547-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786056586788339-29547-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36789
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786056586788339-29547-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36301
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36789
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:54:02.464969 19775 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:54:02.465161 19775 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:54:02.465183 19775 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:54:02.466339 19775 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:54:02.466383 19775 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:54:02.466398 19775 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:54:02.466410 19775 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:54:02.466423 19775 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:54:02.467919 19775 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36301
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786056586788339-29547-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786056586788339-29547-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36789
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786056586788339-29547-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36789
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19775
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786056586788339-29547-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:54:02.468158 19775 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:54:02.468420 19775 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:54:02.473387 19781 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:54:02.473407 19784 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:54:02.473574 19775 server_base.cc:1034] running on GCE node
W20260806 22:54:02.473387 19782 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:54:02.473937 19775 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:54:02.474264 19775 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:54:02.475457 19775 hybrid_clock.cc:648] HybridClock initialized: now 1786056842475432 us; error 36 us; skew 500 ppm
Aug 06 22:54:02 dist-test-slave-1xrh krb5kdc[19761](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056842, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:54:02.478233 19775 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 22:54:02.478726 19775 webserver.cc:460] Webserver started at http://127.0.0.1:35169/ using document root <none> and password file <none>
I20260806 22:54:02.479017 19775 fs_manager.cc:359] Metadata directory not provided
I20260806 22:54:02.479092 19775 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:54:02.479250 19775 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:54:02.480262 19775 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786056586788339-29547-0/minicluster-data/master-0/data/instance:
uuid: "aa57e5a41b8848bba01e0072f3af41d1"
format_stamp: "Formatted at 2026-08-06 22:54:02 on dist-test-slave-1xrh"
server_key: "e979ff0451bc86ad6c9fbf231c38fc88"
server_key_iv: "0d11f707ebfa41d49f1e6176fd430781"
server_key_version: "encryptionkey@0"
I20260806 22:54:02.480573 19775 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786056586788339-29547-0/minicluster-data/master-0/wal/instance:
uuid: "aa57e5a41b8848bba01e0072f3af41d1"
format_stamp: "Formatted at 2026-08-06 22:54:02 on dist-test-slave-1xrh"
server_key: "e979ff0451bc86ad6c9fbf231c38fc88"
server_key_iv: "0d11f707ebfa41d49f1e6176fd430781"
server_key_version: "encryptionkey@0"
I20260806 22:54:02.481877 19775 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:54:02.482573 19791 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:54:02.482750 19775 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:54:02.482889 19775 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786056586788339-29547-0/minicluster-data/master-0/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786056586788339-29547-0/minicluster-data/master-0/wal
uuid: "aa57e5a41b8848bba01e0072f3af41d1"
format_stamp: "Formatted at 2026-08-06 22:54:02 on dist-test-slave-1xrh"
server_key: "e979ff0451bc86ad6c9fbf231c38fc88"
server_key_iv: "0d11f707ebfa41d49f1e6176fd430781"
server_key_version: "encryptionkey@0"
I20260806 22:54:02.482971 19775 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786056586788339-29547-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786056586788339-29547-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786056586788339-29547-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:54:02.486287 19775 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:54:02.488662 19775 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:54:02.488824 19775 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:54:02.493209 19775 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36789
I20260806 22:54:02.493242 19848 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36789 every 8 connection(s)
I20260806 22:54:02.493570 19775 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
I20260806 22:54:02.494555 19849 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:54:02.496829 19849 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa57e5a41b8848bba01e0072f3af41d1: Bootstrap starting.
I20260806 22:54:02.497179 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 19775
I20260806 22:54:02.497326 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786056586788339-29547-0/minicluster-data/master-0/wal/instance
I20260806 22:54:02.497507 19849 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aa57e5a41b8848bba01e0072f3af41d1: Neither blocks nor log segments found. Creating new log.
I20260806 22:54:02.497524 29547 external_mini_cluster.cc:1428] Setting key c353d52e7b96ac8746b595093612d6a2
I20260806 22:54:02.497794 19849 log.cc:824] T 00000000000000000000000000000000 P aa57e5a41b8848bba01e0072f3af41d1: Log is configured to *not* fsync() on all Append() calls
I20260806 22:54:02.498600 19849 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa57e5a41b8848bba01e0072f3af41d1: No bootstrap required, opened a new log
I20260806 22:54:02.500178 19853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:54:02.498088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.186:60419 (local address 127.0.0.1:36789)
0806 22:54:02.498289 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0806 22:54:02.498307 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 22:54:02.498342 (+ 35us) server_negotiation.cc:407] Connection header received
0806 22:54:02.498840 (+ 498us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:54:02.498848 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:54:02.498874 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:54:02.498990 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:54:02.499676 (+ 686us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:54:02.499679 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:54:02.499679 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:54:02.499683 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:54:02.499721 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:54:02.499731 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:54:02.499813 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 22:54:02.499904 (+ 91us) server_negotiation.cc:299] Negotiation successful
0806 22:54:02.499975 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":71,"threads_started":1}
I20260806 22:54:02.501202 29547 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:54:02.501263 19849 raft_consensus.cc:348] T 00000000000000000000000000000000 P aa57e5a41b8848bba01e0072f3af41d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa57e5a41b8848bba01e0072f3af41d1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36789 } }
I20260806 22:54:02.501443 19849 raft_consensus.cc:374] T 00000000000000000000000000000000 P aa57e5a41b8848bba01e0072f3af41d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:54:02.501489 19849 raft_consensus.cc:729] T 00000000000000000000000000000000 P aa57e5a41b8848bba01e0072f3af41d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa57e5a41b8848bba01e0072f3af41d1, State: Initialized, Role: FOLLOWER
I20260806 22:54:02.501607 19849 consensus_queue.cc:260] T 00000000000000000000000000000000 P aa57e5a41b8848bba01e0072f3af41d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa57e5a41b8848bba01e0072f3af41d1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36789 } }
I20260806 22:54:02.501686 19849 raft_consensus.cc:388] T 00000000000000000000000000000000 P aa57e5a41b8848bba01e0072f3af41d1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:54:02.501709 19849 raft_consensus.cc:482] T 00000000000000000000000000000000 P aa57e5a41b8848bba01e0072f3af41d1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:54:02.501773 19849 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aa57e5a41b8848bba01e0072f3af41d1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:54:02.502377 19849 raft_consensus.cc:504] T 00000000000000000000000000000000 P aa57e5a41b8848bba01e0072f3af41d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa57e5a41b8848bba01e0072f3af41d1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36789 } }
I20260806 22:54:02.502506 19849 leader_election.cc:302] T 00000000000000000000000000000000 P aa57e5a41b8848bba01e0072f3af41d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: aa57e5a41b8848bba01e0072f3af41d1; no voters:
I20260806 22:54:02.502671 19849 leader_election.cc:288] T 00000000000000000000000000000000 P aa57e5a41b8848bba01e0072f3af41d1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:54:02.502817 19858 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aa57e5a41b8848bba01e0072f3af41d1 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:54:02.503001 19849 sys_catalog.cc:564] T 00000000000000000000000000000000 P aa57e5a41b8848bba01e0072f3af41d1 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:54:02.503032 19858 raft_consensus.cc:686] T 00000000000000000000000000000000 P aa57e5a41b8848bba01e0072f3af41d1 [term 1 LEADER]: Becoming Leader. State: Replica: aa57e5a41b8848bba01e0072f3af41d1, State: Running, Role: LEADER
I20260806 22:54:02.503270 19858 consensus_queue.cc:237] T 00000000000000000000000000000000 P aa57e5a41b8848bba01e0072f3af41d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa57e5a41b8848bba01e0072f3af41d1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36789 } }
I20260806 22:54:02.503657 19861 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa57e5a41b8848bba01e0072f3af41d1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader aa57e5a41b8848bba01e0072f3af41d1. Latest consensus state: current_term: 1 leader_uuid: "aa57e5a41b8848bba01e0072f3af41d1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa57e5a41b8848bba01e0072f3af41d1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36789 } } }
I20260806 22:54:02.503752 19861 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa57e5a41b8848bba01e0072f3af41d1 [sys.catalog]: This master's current role is: LEADER
I20260806 22:54:02.503674 19860 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa57e5a41b8848bba01e0072f3af41d1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "aa57e5a41b8848bba01e0072f3af41d1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa57e5a41b8848bba01e0072f3af41d1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36789 } } }
I20260806 22:54:02.504000 19860 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa57e5a41b8848bba01e0072f3af41d1 [sys.catalog]: This master's current role is: LEADER
I20260806 22:54:02.503993 19868 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:54:02.504578 19868 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:54:02.505136 19853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:54:02.502729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.186:46647 (local address 127.0.0.1:36789)
0806 22:54:02.502766 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0806 22:54:02.502768 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:54:02.502888 (+ 120us) server_negotiation.cc:407] Connection header received
0806 22:54:02.502988 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:54:02.502989 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:54:02.502994 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:54:02.503025 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:54:02.504738 (+ 1713us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:54:02.504739 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:54:02.504739 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:54:02.504744 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:54:02.504758 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:54:02.504763 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:54:02.504815 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 22:54:02.505044 (+ 229us) server_negotiation.cc:299] Negotiation successful
0806 22:54:02.505060 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 22:54:02.506054 19868 catalog_manager.cc:1347] Generated new cluster ID: 5ec433a4c23747e3a9d6f1a93a44fe36
I20260806 22:54:02.506106 19868 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:54:02.524770 19868 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:54:02.525025 19868 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:54:02.531023 19868 catalog_manager.cc:5868] T 00000000000000000000000000000000 P aa57e5a41b8848bba01e0072f3af41d1: Generated new TSK 0
I20260806 22:54:02.531235 19868 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:54:02.538489 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 19775
2026-08-06T22:54:02Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1354 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786056586788339-29547-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:54:02 dist-test-slave-1xrh krb5kdc[19881](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:54:02 dist-test-slave-1xrh krb5kdc[19881](info): set up 2 sockets
Aug 06 22:54:02 dist-test-slave-1xrh krb5kdc[19881](info): commencing operation
krb5kdc: starting...
W20260806 22:54:03.603730 29547 mini_kdc.cc:121] Time spent starting KDC: real 1.044s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786056586788339-29547-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786056586788339-29547-0/krb5kdc/test-user.keytab.
Aug 06 22:54:03 dist-test-slave-1xrh krb5kdc[19881](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056843, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:54:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:54:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.218.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.218.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.218.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab.
Entry for principal kudu/127.28.218.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab.
I20260806 22:54:03.728312 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786056586788339-29547-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786056586788339-29547-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786056586788339-29547-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.218.254:46241
--webserver_interface=127.28.218.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab
--principal=kudu/127.28.218.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.218.212:37673
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.218.254:46241
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:54:03.812537 19896 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:54:03.812738 19896 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:54:03.812762 19896 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:54:03.813920 19896 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:54:03.813966 19896 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:54:03.813980 19896 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:54:03.813993 19896 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:54:03.814006 19896 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:54:03.815510 19896 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:37673
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786056586788339-29547-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786056586788339-29547-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.218.254:46241
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab
--principal=kudu/127.28.218.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.218.254:46241
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.218.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19896
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786056586788339-29547-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:54:03.815754 19896 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:54:03.815984 19896 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:54:03.821028 19904 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:54:03.821041 19901 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:54:03.821033 19902 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:54:03.821177 19896 server_base.cc:1034] running on GCE node
I20260806 22:54:03.821584 19896 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:54:03.821826 19896 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:54:03.822990 19896 hybrid_clock.cc:648] HybridClock initialized: now 1786056843822976 us; error 30 us; skew 500 ppm
Aug 06 22:54:03 dist-test-slave-1xrh krb5kdc[19881](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056843, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:54:03.830618 19896 init.cc:377] Logged in from keytab as kudu/127.28.218.254@KRBTEST.COM (short username kudu)
I20260806 22:54:03.831059 19896 webserver.cc:460] Webserver started at http://127.28.218.254:42501/ using document root <none> and password file <none>
I20260806 22:54:03.831311 19896 fs_manager.cc:359] Metadata directory not provided
I20260806 22:54:03.831383 19896 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:54:03.831550 19896 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:54:03.832675 19896 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786056586788339-29547-0/minicluster-data/master-0/data/instance:
uuid: "107b0de5a9944b48836999e999a7016c"
format_stamp: "Formatted at 2026-08-06 22:54:03 on dist-test-slave-1xrh"
server_key: "f923f6a714d06832b4a579c5ce75dca1"
server_key_iv: "e4c807243cb0abb8a80972d72d81ea23"
server_key_version: "encryptionkey@0"
I20260806 22:54:03.833055 19896 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786056586788339-29547-0/minicluster-data/master-0/wal/instance:
uuid: "107b0de5a9944b48836999e999a7016c"
format_stamp: "Formatted at 2026-08-06 22:54:03 on dist-test-slave-1xrh"
server_key: "f923f6a714d06832b4a579c5ce75dca1"
server_key_iv: "e4c807243cb0abb8a80972d72d81ea23"
server_key_version: "encryptionkey@0"
I20260806 22:54:03.834383 19896 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 22:54:03.835075 19911 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:54:03.835282 19896 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 22:54:03.835379 19896 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786056586788339-29547-0/minicluster-data/master-0/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786056586788339-29547-0/minicluster-data/master-0/wal
uuid: "107b0de5a9944b48836999e999a7016c"
format_stamp: "Formatted at 2026-08-06 22:54:03 on dist-test-slave-1xrh"
server_key: "f923f6a714d06832b4a579c5ce75dca1"
server_key_iv: "e4c807243cb0abb8a80972d72d81ea23"
server_key_version: "encryptionkey@0"
I20260806 22:54:03.835457 19896 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786056586788339-29547-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786056586788339-29547-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786056586788339-29547-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:54:03.844743 19896 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:54:03.846747 19896 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:54:03.846910 19896 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:54:03.851075 19896 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.254:46241
I20260806 22:54:03.851109 19967 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.254:46241 every 8 connection(s)
I20260806 22:54:03.851517 19896 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
I20260806 22:54:03.852391 19968 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:54:03.853258 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 19896
I20260806 22:54:03.853354 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786056586788339-29547-0/minicluster-data/master-0/wal/instance
I20260806 22:54:03.853574 29547 external_mini_cluster.cc:1428] Setting key d309dc8d3efa42189e8f53efe45ff68b
I20260806 22:54:03.855055 19968 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 107b0de5a9944b48836999e999a7016c: Bootstrap starting.
I20260806 22:54:03.855839 19968 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 107b0de5a9944b48836999e999a7016c: Neither blocks nor log segments found. Creating new log.
I20260806 22:54:03.856153 19968 log.cc:824] T 00000000000000000000000000000000 P 107b0de5a9944b48836999e999a7016c: Log is configured to *not* fsync() on all Append() calls
I20260806 22:54:03.857122 19968 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 107b0de5a9944b48836999e999a7016c: No bootstrap required, opened a new log
I20260806 22:54:03.859686 19968 raft_consensus.cc:348] T 00000000000000000000000000000000 P 107b0de5a9944b48836999e999a7016c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "107b0de5a9944b48836999e999a7016c" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 46241 } }
I20260806 22:54:03.859880 19968 raft_consensus.cc:374] T 00000000000000000000000000000000 P 107b0de5a9944b48836999e999a7016c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:54:03.859921 19968 raft_consensus.cc:729] T 00000000000000000000000000000000 P 107b0de5a9944b48836999e999a7016c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 107b0de5a9944b48836999e999a7016c, State: Initialized, Role: FOLLOWER
I20260806 22:54:03.860072 19968 consensus_queue.cc:260] T 00000000000000000000000000000000 P 107b0de5a9944b48836999e999a7016c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "107b0de5a9944b48836999e999a7016c" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 46241 } }
I20260806 22:54:03.860175 19968 raft_consensus.cc:388] T 00000000000000000000000000000000 P 107b0de5a9944b48836999e999a7016c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:54:03.860213 19968 raft_consensus.cc:482] T 00000000000000000000000000000000 P 107b0de5a9944b48836999e999a7016c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:54:03.860268 19968 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 107b0de5a9944b48836999e999a7016c [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:54:03.860973 19968 raft_consensus.cc:504] T 00000000000000000000000000000000 P 107b0de5a9944b48836999e999a7016c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "107b0de5a9944b48836999e999a7016c" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 46241 } }
I20260806 22:54:03.861068 19968 leader_election.cc:302] T 00000000000000000000000000000000 P 107b0de5a9944b48836999e999a7016c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 107b0de5a9944b48836999e999a7016c; no voters:
Aug 06 22:54:03 dist-test-slave-1xrh krb5kdc[19881](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056843, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.218.254@KRBTEST.COM
I20260806 22:54:03.861241 19968 leader_election.cc:288] T 00000000000000000000000000000000 P 107b0de5a9944b48836999e999a7016c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:54:03.861279 19973 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 107b0de5a9944b48836999e999a7016c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:54:03.861399 19973 raft_consensus.cc:686] T 00000000000000000000000000000000 P 107b0de5a9944b48836999e999a7016c [term 1 LEADER]: Becoming Leader. State: Replica: 107b0de5a9944b48836999e999a7016c, State: Running, Role: LEADER
I20260806 22:54:03.861526 19968 sys_catalog.cc:564] T 00000000000000000000000000000000 P 107b0de5a9944b48836999e999a7016c [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:54:03.861548 19973 consensus_queue.cc:237] T 00000000000000000000000000000000 P 107b0de5a9944b48836999e999a7016c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "107b0de5a9944b48836999e999a7016c" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 46241 } }
I20260806 22:54:03.861871 19975 sys_catalog.cc:455] T 00000000000000000000000000000000 P 107b0de5a9944b48836999e999a7016c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 107b0de5a9944b48836999e999a7016c. Latest consensus state: current_term: 1 leader_uuid: "107b0de5a9944b48836999e999a7016c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "107b0de5a9944b48836999e999a7016c" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 46241 } } }
I20260806 22:54:03.861967 19975 sys_catalog.cc:458] T 00000000000000000000000000000000 P 107b0de5a9944b48836999e999a7016c [sys.catalog]: This master's current role is: LEADER
I20260806 22:54:03.862207 19974 sys_catalog.cc:455] T 00000000000000000000000000000000 P 107b0de5a9944b48836999e999a7016c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "107b0de5a9944b48836999e999a7016c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "107b0de5a9944b48836999e999a7016c" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 46241 } } }
I20260806 22:54:03.862365 19974 sys_catalog.cc:458] T 00000000000000000000000000000000 P 107b0de5a9944b48836999e999a7016c [sys.catalog]: This master's current role is: LEADER
I20260806 22:54:03.862926 19979 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:54:03.863503 19979 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:54:03.865407 19979 catalog_manager.cc:1347] Generated new cluster ID: 0be0ecb35924403986fc1a98dd7d0720
I20260806 22:54:03.865509 19979 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:54:03.865845 19971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:54:03.854089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60643 (local address 127.28.218.254:46241)
0806 22:54:03.854274 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0806 22:54:03.854281 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:54:03.854306 (+ 25us) server_negotiation.cc:407] Connection header received
0806 22:54:03.854815 (+ 509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:54:03.854822 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:54:03.854876 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:54:03.854997 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:54:03.855842 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:54:03.857244 (+ 1402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:54:03.858451 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:54:03.858906 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:54:03.858986 (+ 80us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:54:03.861856 (+ 2870us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:54:03.861860 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:54:03.861900 (+ 40us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:54:03.861915 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:54:03.864096 (+ 2181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:54:03.864575 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:54:03.864578 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:54:03.864578 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:54:03.864677 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:54:03.864956 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:54:03.864957 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:54:03.864958 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:54:03.865187 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:54:03.865341 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0806 22:54:03.865535 (+ 194us) server_negotiation.cc:299] Negotiation successful
0806 22:54:03.865648 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":92,"threads_started":1}
I20260806 22:54:03.866920 29547 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:54:03.875890 19971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:54:03.868461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45509 (local address 127.28.218.254:46241)
0806 22:54:03.868495 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0806 22:54:03.868497 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:54:03.868505 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:54:03.868550 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:54:03.868551 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:54:03.868560 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:54:03.868622 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:54:03.869213 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:54:03.870289 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:54:03.871260 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:54:03.871576 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:54:03.871661 (+ 85us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:54:03.872957 (+ 1296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:54:03.872959 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:54:03.872959 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:54:03.872966 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:54:03.874550 (+ 1584us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:54:03.874951 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:54:03.874952 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:54:03.874952 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:54:03.875013 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:54:03.875237 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:54:03.875237 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:54:03.875237 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:54:03.875366 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:54:03.875468 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0806 22:54:03.875620 (+ 152us) server_negotiation.cc:299] Negotiation successful
0806 22:54:03.875685 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 22:54:03.878850 19979 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:54:03.879441 19979 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:54:03.886864 19979 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 107b0de5a9944b48836999e999a7016c: Generated new TSK 0
I20260806 22:54:03.887041 19979 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:54:03.891858 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 19896
W20260806 22:54:03.898092 19879 connection.cc:535] client connection to 127.28.218.254:46241 recv error: Network error: failed to read from TLS socket (remote: 127.28.218.254:46241): Connection reset by peer (error 104)
2026-08-06T22:54:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1352 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786056586788339-29547-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:54:03 dist-test-slave-1xrh krb5kdc[19999](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:54:03 dist-test-slave-1xrh krb5kdc[19999](info): set up 2 sockets
Aug 06 22:54:03 dist-test-slave-1xrh krb5kdc[19999](info): commencing operation
krb5kdc: starting...
W20260806 22:54:04.948593 29547 mini_kdc.cc:121] Time spent starting KDC: real 1.037s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786056586788339-29547-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786056586788339-29547-0/krb5kdc/test-user.keytab.
Aug 06 22:54:05 dist-test-slave-1xrh krb5kdc[19999](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056845, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:54:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:54:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.218.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.218.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.218.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab.
Entry for principal kudu/127.28.218.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab.
I20260806 22:54:05.074435 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786056586788339-29547-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786056586788339-29547-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786056586788339-29547-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.218.254:34951
--webserver_interface=127.28.218.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab
--principal=kudu/127.28.218.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.218.212:32907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.218.254:34951
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:54:05.157394 20012 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:54:05.157724 20012 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:54:05.157775 20012 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:54:05.159231 20012 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:54:05.159334 20012 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:54:05.159358 20012 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:54:05.159390 20012 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:54:05.159422 20012 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:54:05.161382 20012 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:32907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786056586788339-29547-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786056586788339-29547-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.218.254:34951
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab
--principal=kudu/127.28.218.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.218.254:34951
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.218.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20012
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786056586788339-29547-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:54:05.161799 20012 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:54:05.162109 20012 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:54:05.167667 20018 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:54:05.167670 20020 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:54:05.167686 20017 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:54:05.167853 20012 server_base.cc:1034] running on GCE node
I20260806 22:54:05.168203 20012 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:54:05.168495 20012 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:54:05.169652 20012 hybrid_clock.cc:648] HybridClock initialized: now 1786056845169646 us; error 53 us; skew 500 ppm
Aug 06 22:54:05 dist-test-slave-1xrh krb5kdc[19999](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056845, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:54:05.172396 20012 init.cc:377] Logged in from keytab as kudu/127.28.218.254@KRBTEST.COM (short username kudu)
I20260806 22:54:05.172904 20012 webserver.cc:460] Webserver started at http://127.28.218.254:46523/ using document root <none> and password file <none>
I20260806 22:54:05.173148 20012 fs_manager.cc:359] Metadata directory not provided
I20260806 22:54:05.173216 20012 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:54:05.173379 20012 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:54:05.174508 20012 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786056586788339-29547-0/minicluster-data/master-0/data/instance:
uuid: "9f56511f7a974ef695c34ad984a1b7e7"
format_stamp: "Formatted at 2026-08-06 22:54:05 on dist-test-slave-1xrh"
server_key: "7c37fecd20ecd9ac3ee66c33c1a822bc"
server_key_iv: "6992d44ebdaa4a67664cb30fc2036697"
server_key_version: "encryptionkey@0"
I20260806 22:54:05.174890 20012 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786056586788339-29547-0/minicluster-data/master-0/wal/instance:
uuid: "9f56511f7a974ef695c34ad984a1b7e7"
format_stamp: "Formatted at 2026-08-06 22:54:05 on dist-test-slave-1xrh"
server_key: "7c37fecd20ecd9ac3ee66c33c1a822bc"
server_key_iv: "6992d44ebdaa4a67664cb30fc2036697"
server_key_version: "encryptionkey@0"
I20260806 22:54:05.176383 20012 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:54:05.177119 20028 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:54:05.177286 20012 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:54:05.177398 20012 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786056586788339-29547-0/minicluster-data/master-0/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786056586788339-29547-0/minicluster-data/master-0/wal
uuid: "9f56511f7a974ef695c34ad984a1b7e7"
format_stamp: "Formatted at 2026-08-06 22:54:05 on dist-test-slave-1xrh"
server_key: "7c37fecd20ecd9ac3ee66c33c1a822bc"
server_key_iv: "6992d44ebdaa4a67664cb30fc2036697"
server_key_version: "encryptionkey@0"
I20260806 22:54:05.177469 20012 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786056586788339-29547-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786056586788339-29547-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786056586788339-29547-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:54:05.187793 20012 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:54:05.189514 20012 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:54:05.189678 20012 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:54:05.193800 20012 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.254:34951
I20260806 22:54:05.193887 20080 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.254:34951 every 8 connection(s)
I20260806 22:54:05.194211 20012 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
I20260806 22:54:05.195029 20081 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:54:05.197364 20081 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f56511f7a974ef695c34ad984a1b7e7: Bootstrap starting.
I20260806 22:54:05.197883 20081 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9f56511f7a974ef695c34ad984a1b7e7: Neither blocks nor log segments found. Creating new log.
I20260806 22:54:05.198117 20081 log.cc:824] T 00000000000000000000000000000000 P 9f56511f7a974ef695c34ad984a1b7e7: Log is configured to *not* fsync() on all Append() calls
I20260806 22:54:05.198777 20081 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f56511f7a974ef695c34ad984a1b7e7: No bootstrap required, opened a new log
I20260806 22:54:05.199613 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 20012
I20260806 22:54:05.199738 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786056586788339-29547-0/minicluster-data/master-0/wal/instance
I20260806 22:54:05.199935 29547 external_mini_cluster.cc:1428] Setting key 561dd4e70ac6f38614cc4619eb820896
I20260806 22:54:05.201555 20081 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9f56511f7a974ef695c34ad984a1b7e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f56511f7a974ef695c34ad984a1b7e7" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 34951 } }
I20260806 22:54:05.201735 20081 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9f56511f7a974ef695c34ad984a1b7e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:54:05.201777 20081 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9f56511f7a974ef695c34ad984a1b7e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f56511f7a974ef695c34ad984a1b7e7, State: Initialized, Role: FOLLOWER
I20260806 22:54:05.201905 20081 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9f56511f7a974ef695c34ad984a1b7e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f56511f7a974ef695c34ad984a1b7e7" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 34951 } }
I20260806 22:54:05.201992 20081 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9f56511f7a974ef695c34ad984a1b7e7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:54:05.202034 20081 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9f56511f7a974ef695c34ad984a1b7e7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:54:05.202101 20081 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9f56511f7a974ef695c34ad984a1b7e7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:54:05.202629 20081 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9f56511f7a974ef695c34ad984a1b7e7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f56511f7a974ef695c34ad984a1b7e7" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 34951 } }
I20260806 22:54:05.202746 20081 leader_election.cc:302] T 00000000000000000000000000000000 P 9f56511f7a974ef695c34ad984a1b7e7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9f56511f7a974ef695c34ad984a1b7e7; no voters:
I20260806 22:54:05.202940 20081 leader_election.cc:288] T 00000000000000000000000000000000 P 9f56511f7a974ef695c34ad984a1b7e7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:54:05.202971 20086 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9f56511f7a974ef695c34ad984a1b7e7 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:54:05.203107 20086 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9f56511f7a974ef695c34ad984a1b7e7 [term 1 LEADER]: Becoming Leader. State: Replica: 9f56511f7a974ef695c34ad984a1b7e7, State: Running, Role: LEADER
I20260806 22:54:05.203236 20081 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9f56511f7a974ef695c34ad984a1b7e7 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:54:05.203284 20086 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9f56511f7a974ef695c34ad984a1b7e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f56511f7a974ef695c34ad984a1b7e7" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 34951 } }
I20260806 22:54:05.203688 20087 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f56511f7a974ef695c34ad984a1b7e7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9f56511f7a974ef695c34ad984a1b7e7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f56511f7a974ef695c34ad984a1b7e7" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 34951 } } }
I20260806 22:54:05.203789 20087 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f56511f7a974ef695c34ad984a1b7e7 [sys.catalog]: This master's current role is: LEADER
I20260806 22:54:05.203680 20086 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f56511f7a974ef695c34ad984a1b7e7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9f56511f7a974ef695c34ad984a1b7e7. Latest consensus state: current_term: 1 leader_uuid: "9f56511f7a974ef695c34ad984a1b7e7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f56511f7a974ef695c34ad984a1b7e7" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 34951 } } }
I20260806 22:54:05.203863 20086 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f56511f7a974ef695c34ad984a1b7e7 [sys.catalog]: This master's current role is: LEADER
W20260806 22:54:05.205050 20102 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9f56511f7a974ef695c34ad984a1b7e7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 22:54:05.205119 20102 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 22:54:05.205209 20094 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:54:05.205403 20094 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:54:05.206563 20085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:54:05.200444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56771 (local address 127.28.218.254:34951)
0806 22:54:05.200610 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0806 22:54:05.200617 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:54:05.200672 (+ 55us) server_negotiation.cc:407] Connection header received
0806 22:54:05.201487 (+ 815us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:54:05.201494 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:54:05.201538 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:54:05.201647 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:54:05.201963 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:54:05.203052 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:54:05.204192 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:54:05.204953 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:54:05.205392 (+ 439us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:54:05.205844 (+ 452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:54:05.205849 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:54:05.205849 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:54:05.206070 (+ 221us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:54:05.206112 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:54:05.206123 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:54:05.206225 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0806 22:54:05.206314 (+ 89us) server_negotiation.cc:299] Negotiation successful
0806 22:54:05.206342 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":93,"threads_started":1}
I20260806 22:54:05.206979 20094 catalog_manager.cc:1347] Generated new cluster ID: ba7a002fab394447b3a491abd005e041
I20260806 22:54:05.207029 20094 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:54:05.207620 29547 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:54:05.214507 20085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:54:05.211485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57511 (local address 127.28.218.254:34951)
0806 22:54:05.211523 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0806 22:54:05.211525 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:54:05.211707 (+ 182us) server_negotiation.cc:407] Connection header received
0806 22:54:05.211784 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:54:05.211785 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:54:05.211795 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:54:05.211857 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:54:05.212024 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:54:05.212500 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:54:05.213264 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:54:05.213468 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:54:05.213547 (+ 79us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:54:05.214247 (+ 700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:54:05.214248 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:54:05.214249 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:54:05.214254 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:54:05.214267 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:54:05.214272 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:54:05.214345 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 22:54:05.214420 (+ 75us) server_negotiation.cc:299] Negotiation successful
0806 22:54:05.214429 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 22:54:05.216353 20094 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:54:05.217048 20094 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:54:05.223918 20094 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9f56511f7a974ef695c34ad984a1b7e7: Generated new TSK 0
I20260806 22:54:05.224164 20094 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:54:05.228523 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 20012
W20260806 22:54:05.236191 19997 connection.cc:535] client connection to 127.28.218.254:34951 recv error: Network error: failed to read from TLS socket (remote: 127.28.218.254:34951): Connection reset by peer (error 104)
2026-08-06T22:54:05Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1338 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786056586788339-29547-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:54:05 dist-test-slave-1xrh krb5kdc[20113](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:54:05 dist-test-slave-1xrh krb5kdc[20113](info): set up 2 sockets
Aug 06 22:54:05 dist-test-slave-1xrh krb5kdc[20113](info): commencing operation
krb5kdc: starting...
W20260806 22:54:06.332226 29547 mini_kdc.cc:121] Time spent starting KDC: real 1.082s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786056586788339-29547-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786056586788339-29547-0/krb5kdc/test-user.keytab.
Aug 06 22:54:06 dist-test-slave-1xrh krb5kdc[20113](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056846, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:54:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:54:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.218.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.218.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.218.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab.
Entry for principal kudu/127.28.218.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab.
I20260806 22:54:06.458876 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786056586788339-29547-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786056586788339-29547-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786056586788339-29547-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.218.254:40333
--webserver_interface=127.28.218.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab
--principal=kudu/127.28.218.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.218.212:38849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.218.254:40333
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:54:06.540120 20126 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:54:06.540336 20126 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:54:06.540370 20126 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:54:06.541451 20126 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:54:06.541514 20126 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:54:06.541536 20126 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:54:06.541556 20126 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:54:06.541582 20126 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:54:06.542980 20126 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:38849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786056586788339-29547-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786056586788339-29547-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.218.254:40333
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab
--principal=kudu/127.28.218.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.218.254:40333
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.218.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20126
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786056586788339-29547-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:54:06.543260 20126 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:54:06.543488 20126 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:54:06.548123 20132 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:54:06.548221 20126 server_base.cc:1034] running on GCE node
W20260806 22:54:06.548130 20134 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:54:06.548130 20131 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:54:06.548555 20126 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:54:06.548851 20126 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:54:06.550010 20126 hybrid_clock.cc:648] HybridClock initialized: now 1786056846550016 us; error 41 us; skew 500 ppm
Aug 06 22:54:06 dist-test-slave-1xrh krb5kdc[20113](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056846, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:54:06.552379 20126 init.cc:377] Logged in from keytab as kudu/127.28.218.254@KRBTEST.COM (short username kudu)
I20260806 22:54:06.552799 20126 webserver.cc:460] Webserver started at http://127.28.218.254:34747/ using document root <none> and password file <none>
I20260806 22:54:06.553036 20126 fs_manager.cc:359] Metadata directory not provided
I20260806 22:54:06.553105 20126 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:54:06.553256 20126 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:54:06.554284 20126 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786056586788339-29547-0/minicluster-data/master-0/data/instance:
uuid: "677f4c5ce5074dd79d274141f3857fbd"
format_stamp: "Formatted at 2026-08-06 22:54:06 on dist-test-slave-1xrh"
server_key: "11f6fa4a78eee0fe351aa64520471141"
server_key_iv: "7b75b7ca28680acb81930390f58c21e7"
server_key_version: "encryptionkey@0"
I20260806 22:54:06.554657 20126 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786056586788339-29547-0/minicluster-data/master-0/wal/instance:
uuid: "677f4c5ce5074dd79d274141f3857fbd"
format_stamp: "Formatted at 2026-08-06 22:54:06 on dist-test-slave-1xrh"
server_key: "11f6fa4a78eee0fe351aa64520471141"
server_key_iv: "7b75b7ca28680acb81930390f58c21e7"
server_key_version: "encryptionkey@0"
I20260806 22:54:06.556001 20126 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:54:06.556689 20141 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:54:06.556844 20126 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:54:06.556975 20126 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786056586788339-29547-0/minicluster-data/master-0/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786056586788339-29547-0/minicluster-data/master-0/wal
uuid: "677f4c5ce5074dd79d274141f3857fbd"
format_stamp: "Formatted at 2026-08-06 22:54:06 on dist-test-slave-1xrh"
server_key: "11f6fa4a78eee0fe351aa64520471141"
server_key_iv: "7b75b7ca28680acb81930390f58c21e7"
server_key_version: "encryptionkey@0"
I20260806 22:54:06.557049 20126 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786056586788339-29547-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786056586788339-29547-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786056586788339-29547-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:54:06.560189 20126 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:54:06.561905 20126 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:54:06.562078 20126 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:54:06.566074 20126 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.254:40333
I20260806 22:54:06.566109 20196 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.254:40333 every 8 connection(s)
I20260806 22:54:06.566409 20126 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
I20260806 22:54:06.567386 20197 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:54:06.569394 20197 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 677f4c5ce5074dd79d274141f3857fbd: Bootstrap starting.
I20260806 22:54:06.569938 20197 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 677f4c5ce5074dd79d274141f3857fbd: Neither blocks nor log segments found. Creating new log.
I20260806 22:54:06.570235 20197 log.cc:824] T 00000000000000000000000000000000 P 677f4c5ce5074dd79d274141f3857fbd: Log is configured to *not* fsync() on all Append() calls
I20260806 22:54:06.570887 20197 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 677f4c5ce5074dd79d274141f3857fbd: No bootstrap required, opened a new log
I20260806 22:54:06.572402 20197 raft_consensus.cc:348] T 00000000000000000000000000000000 P 677f4c5ce5074dd79d274141f3857fbd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "677f4c5ce5074dd79d274141f3857fbd" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 40333 } }
I20260806 22:54:06.572536 20197 raft_consensus.cc:374] T 00000000000000000000000000000000 P 677f4c5ce5074dd79d274141f3857fbd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:54:06.572582 20197 raft_consensus.cc:729] T 00000000000000000000000000000000 P 677f4c5ce5074dd79d274141f3857fbd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 677f4c5ce5074dd79d274141f3857fbd, State: Initialized, Role: FOLLOWER
I20260806 22:54:06.572732 20197 consensus_queue.cc:260] T 00000000000000000000000000000000 P 677f4c5ce5074dd79d274141f3857fbd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "677f4c5ce5074dd79d274141f3857fbd" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 40333 } }
I20260806 22:54:06.572831 20197 raft_consensus.cc:388] T 00000000000000000000000000000000 P 677f4c5ce5074dd79d274141f3857fbd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:54:06.572878 20197 raft_consensus.cc:482] T 00000000000000000000000000000000 P 677f4c5ce5074dd79d274141f3857fbd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:54:06.572927 20197 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 677f4c5ce5074dd79d274141f3857fbd [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:54:06.573720 20197 raft_consensus.cc:504] T 00000000000000000000000000000000 P 677f4c5ce5074dd79d274141f3857fbd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "677f4c5ce5074dd79d274141f3857fbd" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 40333 } }
I20260806 22:54:06.573830 20197 leader_election.cc:302] T 00000000000000000000000000000000 P 677f4c5ce5074dd79d274141f3857fbd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 677f4c5ce5074dd79d274141f3857fbd; no voters:
I20260806 22:54:06.574025 20197 leader_election.cc:288] T 00000000000000000000000000000000 P 677f4c5ce5074dd79d274141f3857fbd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:54:06.574086 20200 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 677f4c5ce5074dd79d274141f3857fbd [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:54:06.574285 20200 raft_consensus.cc:686] T 00000000000000000000000000000000 P 677f4c5ce5074dd79d274141f3857fbd [term 1 LEADER]: Becoming Leader. State: Replica: 677f4c5ce5074dd79d274141f3857fbd, State: Running, Role: LEADER
I20260806 22:54:06.574306 20197 sys_catalog.cc:564] T 00000000000000000000000000000000 P 677f4c5ce5074dd79d274141f3857fbd [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:54:06.574375 20200 consensus_queue.cc:237] T 00000000000000000000000000000000 P 677f4c5ce5074dd79d274141f3857fbd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "677f4c5ce5074dd79d274141f3857fbd" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 40333 } }
I20260806 22:54:06.574769 20201 sys_catalog.cc:455] T 00000000000000000000000000000000 P 677f4c5ce5074dd79d274141f3857fbd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "677f4c5ce5074dd79d274141f3857fbd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "677f4c5ce5074dd79d274141f3857fbd" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 40333 } } }
I20260806 22:54:06.574918 20201 sys_catalog.cc:458] T 00000000000000000000000000000000 P 677f4c5ce5074dd79d274141f3857fbd [sys.catalog]: This master's current role is: LEADER
I20260806 22:54:06.575011 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 20126
I20260806 22:54:06.575162 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786056586788339-29547-0/minicluster-data/master-0/wal/instance
I20260806 22:54:06.575213 20209 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:54:06.575359 29547 external_mini_cluster.cc:1428] Setting key 3bdcd06052c4cad41f308c6f0a6d3b6b
I20260806 22:54:06.575736 20209 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:54:06.574770 20202 sys_catalog.cc:455] T 00000000000000000000000000000000 P 677f4c5ce5074dd79d274141f3857fbd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 677f4c5ce5074dd79d274141f3857fbd. Latest consensus state: current_term: 1 leader_uuid: "677f4c5ce5074dd79d274141f3857fbd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "677f4c5ce5074dd79d274141f3857fbd" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 40333 } } }
I20260806 22:54:06.576190 20202 sys_catalog.cc:458] T 00000000000000000000000000000000 P 677f4c5ce5074dd79d274141f3857fbd [sys.catalog]: This master's current role is: LEADER
I20260806 22:54:06.577723 20209 catalog_manager.cc:1347] Generated new cluster ID: ce7a7c31b6a445fe93432179a0054a57
I20260806 22:54:06.577777 20209 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:54:06.578015 20220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:54:06.575967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53255 (local address 127.28.218.254:40333)
0806 22:54:06.576571 (+ 604us) server_negotiation.cc:206] Beginning negotiation
0806 22:54:06.576579 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:54:06.576608 (+ 29us) server_negotiation.cc:407] Connection header received
0806 22:54:06.576712 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:54:06.576718 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:54:06.576746 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:54:06.576905 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:54:06.577559 (+ 654us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:54:06.577562 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:54:06.577562 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:54:06.577567 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:54:06.577614 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:54:06.577628 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:54:06.577697 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 22:54:06.577801 (+ 104us) server_negotiation.cc:299] Negotiation successful
0806 22:54:06.577845 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":558,"thread_start_us":88,"threads_started":1}
I20260806 22:54:06.578895 29547 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:54:06.581424 20220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:54:06.580337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57305 (local address 127.28.218.254:40333)
0806 22:54:06.580373 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0806 22:54:06.580375 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:54:06.580520 (+ 145us) server_negotiation.cc:407] Connection header received
0806 22:54:06.580553 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:54:06.580553 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:54:06.580559 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:54:06.580627 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:54:06.581202 (+ 575us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:54:06.581203 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:54:06.581203 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:54:06.581207 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:54:06.581221 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:54:06.581226 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:54:06.581286 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 22:54:06.581346 (+ 60us) server_negotiation.cc:299] Negotiation successful
0806 22:54:06.581360 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 22:54:06.582675 20209 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:54:06.582800 20209 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:54:06.588656 20209 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 677f4c5ce5074dd79d274141f3857fbd: Generated new TSK 0
I20260806 22:54:06.588811 20209 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:54:06.593075 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 20126
2026-08-06T22:54:06Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1362 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786056586788339-29547-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:54:06 dist-test-slave-1xrh krb5kdc[20229](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:54:06 dist-test-slave-1xrh krb5kdc[20229](info): set up 2 sockets
Aug 06 22:54:06 dist-test-slave-1xrh krb5kdc[20229](info): commencing operation
krb5kdc: starting...
W20260806 22:54:07.642838 29547 mini_kdc.cc:121] Time spent starting KDC: real 1.030s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786056586788339-29547-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786056586788339-29547-0/krb5kdc/test-user.keytab.
Aug 06 22:54:07 dist-test-slave-1xrh krb5kdc[20229](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056847, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:54:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:54:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.218.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.218.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.218.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab.
Entry for principal kudu/127.28.218.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab.
I20260806 22:54:07.757333 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786056586788339-29547-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786056586788339-29547-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786056586788339-29547-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.218.254:34413
--webserver_interface=127.28.218.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab
--principal=kudu/127.28.218.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.218.212:42937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.218.254:34413
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:54:07.838338 20243 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:54:07.838560 20243 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:54:07.838598 20243 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:54:07.839696 20243 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:54:07.839761 20243 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:54:07.839783 20243 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:54:07.839803 20243 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:54:07.839830 20243 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:54:07.841254 20243 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:42937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786056586788339-29547-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786056586788339-29547-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.218.254:34413
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab
--principal=kudu/127.28.218.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.218.254:34413
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.218.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20243
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786056586788339-29547-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:54:07.841522 20243 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:54:07.841732 20243 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:54:07.846516 20249 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:54:07.846541 20252 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:54:07.846596 20243 server_base.cc:1034] running on GCE node
W20260806 22:54:07.846541 20250 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:54:07.846922 20243 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:54:07.847258 20243 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:54:07.848429 20243 hybrid_clock.cc:648] HybridClock initialized: now 1786056847848416 us; error 29 us; skew 500 ppm
Aug 06 22:54:07 dist-test-slave-1xrh krb5kdc[20229](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056847, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:54:07.850884 20243 init.cc:377] Logged in from keytab as kudu/127.28.218.254@KRBTEST.COM (short username kudu)
I20260806 22:54:07.851351 20243 webserver.cc:460] Webserver started at http://127.28.218.254:38279/ using document root <none> and password file <none>
I20260806 22:54:07.851518 20243 fs_manager.cc:359] Metadata directory not provided
I20260806 22:54:07.851557 20243 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:54:07.851640 20243 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:54:07.852545 20243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786056586788339-29547-0/minicluster-data/master-0/data/instance:
uuid: "40f4978221354929b8aea024ab96be95"
format_stamp: "Formatted at 2026-08-06 22:54:07 on dist-test-slave-1xrh"
server_key: "f04d92adfc01ab4fee5b5922276aa638"
server_key_iv: "edd57d7b13f05e1b852b4b39161f012b"
server_key_version: "encryptionkey@0"
I20260806 22:54:07.852823 20243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786056586788339-29547-0/minicluster-data/master-0/wal/instance:
uuid: "40f4978221354929b8aea024ab96be95"
format_stamp: "Formatted at 2026-08-06 22:54:07 on dist-test-slave-1xrh"
server_key: "f04d92adfc01ab4fee5b5922276aa638"
server_key_iv: "edd57d7b13f05e1b852b4b39161f012b"
server_key_version: "encryptionkey@0"
I20260806 22:54:07.854049 20243 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:54:07.854744 20259 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:54:07.854918 20243 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:54:07.855011 20243 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786056586788339-29547-0/minicluster-data/master-0/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786056586788339-29547-0/minicluster-data/master-0/wal
uuid: "40f4978221354929b8aea024ab96be95"
format_stamp: "Formatted at 2026-08-06 22:54:07 on dist-test-slave-1xrh"
server_key: "f04d92adfc01ab4fee5b5922276aa638"
server_key_iv: "edd57d7b13f05e1b852b4b39161f012b"
server_key_version: "encryptionkey@0"
I20260806 22:54:07.855064 20243 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786056586788339-29547-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786056586788339-29547-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786056586788339-29547-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:54:07.873749 20243 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:54:07.875950 20243 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:54:07.876052 20243 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:54:07.879755 20243 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.254:34413
I20260806 22:54:07.879802 20312 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.254:34413 every 8 connection(s)
I20260806 22:54:07.880125 20243 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
I20260806 22:54:07.881012 20313 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:54:07.881517 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 20243
I20260806 22:54:07.881644 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786056586788339-29547-0/minicluster-data/master-0/wal/instance
I20260806 22:54:07.881841 29547 external_mini_cluster.cc:1428] Setting key da67b887d62b8165c47173080d408c12
I20260806 22:54:07.883934 20313 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 40f4978221354929b8aea024ab96be95: Bootstrap starting.
I20260806 22:54:07.884503 20313 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 40f4978221354929b8aea024ab96be95: Neither blocks nor log segments found. Creating new log.
I20260806 22:54:07.884807 20313 log.cc:824] T 00000000000000000000000000000000 P 40f4978221354929b8aea024ab96be95: Log is configured to *not* fsync() on all Append() calls
I20260806 22:54:07.885622 20313 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 40f4978221354929b8aea024ab96be95: No bootstrap required, opened a new log
I20260806 22:54:07.888283 20313 raft_consensus.cc:348] T 00000000000000000000000000000000 P 40f4978221354929b8aea024ab96be95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40f4978221354929b8aea024ab96be95" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 34413 } }
I20260806 22:54:07.888414 20313 raft_consensus.cc:374] T 00000000000000000000000000000000 P 40f4978221354929b8aea024ab96be95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:54:07.888437 20313 raft_consensus.cc:729] T 00000000000000000000000000000000 P 40f4978221354929b8aea024ab96be95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40f4978221354929b8aea024ab96be95, State: Initialized, Role: FOLLOWER
I20260806 22:54:07.888583 20313 consensus_queue.cc:260] T 00000000000000000000000000000000 P 40f4978221354929b8aea024ab96be95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40f4978221354929b8aea024ab96be95" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 34413 } }
I20260806 22:54:07.888643 20313 raft_consensus.cc:388] T 00000000000000000000000000000000 P 40f4978221354929b8aea024ab96be95 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:54:07.888665 20313 raft_consensus.cc:482] T 00000000000000000000000000000000 P 40f4978221354929b8aea024ab96be95 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:54:07.888731 20313 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 40f4978221354929b8aea024ab96be95 [term 0 FOLLOWER]: Advancing to term 1
Aug 06 22:54:07 dist-test-slave-1xrh krb5kdc[20229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056847, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.218.254@KRBTEST.COM
I20260806 22:54:07.889359 20313 raft_consensus.cc:504] T 00000000000000000000000000000000 P 40f4978221354929b8aea024ab96be95 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40f4978221354929b8aea024ab96be95" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 34413 } }
I20260806 22:54:07.889487 20313 leader_election.cc:302] T 00000000000000000000000000000000 P 40f4978221354929b8aea024ab96be95 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 40f4978221354929b8aea024ab96be95; no voters:
I20260806 22:54:07.889672 20313 leader_election.cc:288] T 00000000000000000000000000000000 P 40f4978221354929b8aea024ab96be95 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:54:07.889890 20319 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 40f4978221354929b8aea024ab96be95 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:54:07.889999 20313 sys_catalog.cc:564] T 00000000000000000000000000000000 P 40f4978221354929b8aea024ab96be95 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:54:07.890060 20319 raft_consensus.cc:686] T 00000000000000000000000000000000 P 40f4978221354929b8aea024ab96be95 [term 1 LEADER]: Becoming Leader. State: Replica: 40f4978221354929b8aea024ab96be95, State: Running, Role: LEADER
I20260806 22:54:07.890158 20319 consensus_queue.cc:237] T 00000000000000000000000000000000 P 40f4978221354929b8aea024ab96be95 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40f4978221354929b8aea024ab96be95" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 34413 } }
I20260806 22:54:07.890494 20319 sys_catalog.cc:455] T 00000000000000000000000000000000 P 40f4978221354929b8aea024ab96be95 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 40f4978221354929b8aea024ab96be95. Latest consensus state: current_term: 1 leader_uuid: "40f4978221354929b8aea024ab96be95" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40f4978221354929b8aea024ab96be95" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 34413 } } }
I20260806 22:54:07.890607 20319 sys_catalog.cc:458] T 00000000000000000000000000000000 P 40f4978221354929b8aea024ab96be95 [sys.catalog]: This master's current role is: LEADER
I20260806 22:54:07.890506 20318 sys_catalog.cc:455] T 00000000000000000000000000000000 P 40f4978221354929b8aea024ab96be95 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "40f4978221354929b8aea024ab96be95" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40f4978221354929b8aea024ab96be95" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 34413 } } }
I20260806 22:54:07.890821 20318 sys_catalog.cc:458] T 00000000000000000000000000000000 P 40f4978221354929b8aea024ab96be95 [sys.catalog]: This master's current role is: LEADER
I20260806 22:54:07.890964 20327 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:54:07.891541 20327 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:54:07.893301 20316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:54:07.882336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46583 (local address 127.28.218.254:34413)
0806 22:54:07.882502 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0806 22:54:07.882509 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:54:07.882532 (+ 23us) server_negotiation.cc:407] Connection header received
0806 22:54:07.882906 (+ 374us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:54:07.882912 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:54:07.882968 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:54:07.883075 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:54:07.883967 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:54:07.884890 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:54:07.885978 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:54:07.886354 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:54:07.886431 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:54:07.889323 (+ 2892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:54:07.889328 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:54:07.889328 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:54:07.889342 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:54:07.891647 (+ 2305us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:54:07.892155 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:54:07.892158 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:54:07.892158 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:54:07.892256 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:54:07.892475 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:54:07.892475 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:54:07.892476 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:54:07.892706 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:54:07.892871 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0806 22:54:07.892985 (+ 114us) server_negotiation.cc:299] Negotiation successful
0806 22:54:07.893127 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":91,"threads_started":1}
I20260806 22:54:07.894306 29547 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:54:07.896283 20327 catalog_manager.cc:1347] Generated new cluster ID: 396348cd6ad446d190c584ce3b6fc162
I20260806 22:54:07.896337 20327 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:54:07.901678 20316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:54:07.895654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48939 (local address 127.28.218.254:34413)
0806 22:54:07.895686 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0806 22:54:07.895688 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:54:07.895846 (+ 158us) server_negotiation.cc:407] Connection header received
0806 22:54:07.895869 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:54:07.895869 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:54:07.895879 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:54:07.895930 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:54:07.896544 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:54:07.897033 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:54:07.897732 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:54:07.897960 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:54:07.898035 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:54:07.899175 (+ 1140us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:54:07.899176 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:54:07.899177 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:54:07.899183 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:54:07.900558 (+ 1375us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:54:07.900905 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:54:07.900906 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:54:07.900906 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:54:07.900973 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:54:07.901183 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:54:07.901184 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:54:07.901184 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:54:07.901290 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:54:07.901381 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 22:54:07.901512 (+ 131us) server_negotiation.cc:299] Negotiation successful
0806 22:54:07.901586 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260806 22:54:07.912618 20327 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:54:07.913293 20327 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:54:07.917665 20327 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 40f4978221354929b8aea024ab96be95: Generated new TSK 0
I20260806 22:54:07.917824 20327 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:54:07.926896 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 20243
2026-08-06T22:54:07Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1333 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786056586788339-29547-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:54:07 dist-test-slave-1xrh krb5kdc[20344](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:54:07 dist-test-slave-1xrh krb5kdc[20344](info): set up 2 sockets
Aug 06 22:54:07 dist-test-slave-1xrh krb5kdc[20344](info): commencing operation
krb5kdc: starting...
W20260806 22:54:08.970417 29547 mini_kdc.cc:121] Time spent starting KDC: real 1.024s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786056586788339-29547-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786056586788339-29547-0/krb5kdc/test-user.keytab.
Aug 06 22:54:09 dist-test-slave-1xrh krb5kdc[20344](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056849, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:54:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:54:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.218.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.218.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.218.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab.
Entry for principal kudu/127.28.218.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab.
I20260806 22:54:09.088511 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786056586788339-29547-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786056586788339-29547-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786056586788339-29547-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.218.254:36617
--webserver_interface=127.28.218.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab
--principal=kudu/127.28.218.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.218.212:44393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.218.254:36617
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:54:09.170281 20357 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:54:09.170526 20357 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:54:09.170567 20357 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:54:09.171727 20357 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:54:09.171815 20357 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:54:09.171840 20357 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:54:09.171868 20357 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:54:09.171898 20357 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:54:09.173332 20357 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:44393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786056586788339-29547-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786056586788339-29547-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.218.254:36617
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab
--principal=kudu/127.28.218.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.218.254:36617
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.218.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20357
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786056586788339-29547-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:54:09.173591 20357 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:54:09.173817 20357 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:54:09.178656 20363 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:54:09.178639 20365 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:54:09.178669 20357 server_base.cc:1034] running on GCE node
W20260806 22:54:09.178628 20362 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:54:09.179059 20357 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:54:09.179373 20357 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:54:09.180536 20357 hybrid_clock.cc:648] HybridClock initialized: now 1786056849180531 us; error 35 us; skew 500 ppm
Aug 06 22:54:09 dist-test-slave-1xrh krb5kdc[20344](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056849, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:54:09.187208 20357 init.cc:377] Logged in from keytab as kudu/127.28.218.254@KRBTEST.COM (short username kudu)
I20260806 22:54:09.187659 20357 webserver.cc:460] Webserver started at http://127.28.218.254:37587/ using document root <none> and password file <none>
I20260806 22:54:09.187865 20357 fs_manager.cc:359] Metadata directory not provided
I20260806 22:54:09.187908 20357 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:54:09.188054 20357 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:54:09.189046 20357 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786056586788339-29547-0/minicluster-data/master-0/data/instance:
uuid: "9f1e0f110f8a47c187c9fe4708a3c5ec"
format_stamp: "Formatted at 2026-08-06 22:54:09 on dist-test-slave-1xrh"
server_key: "e6c5442695918fc6fbfad4333edc96c4"
server_key_iv: "abed0bef0dfaa3f140cf4178dbfe316b"
server_key_version: "encryptionkey@0"
I20260806 22:54:09.189402 20357 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786056586788339-29547-0/minicluster-data/master-0/wal/instance:
uuid: "9f1e0f110f8a47c187c9fe4708a3c5ec"
format_stamp: "Formatted at 2026-08-06 22:54:09 on dist-test-slave-1xrh"
server_key: "e6c5442695918fc6fbfad4333edc96c4"
server_key_iv: "abed0bef0dfaa3f140cf4178dbfe316b"
server_key_version: "encryptionkey@0"
I20260806 22:54:09.190675 20357 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:54:09.191452 20372 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:54:09.191641 20357 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:54:09.191710 20357 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786056586788339-29547-0/minicluster-data/master-0/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786056586788339-29547-0/minicluster-data/master-0/wal
uuid: "9f1e0f110f8a47c187c9fe4708a3c5ec"
format_stamp: "Formatted at 2026-08-06 22:54:09 on dist-test-slave-1xrh"
server_key: "e6c5442695918fc6fbfad4333edc96c4"
server_key_iv: "abed0bef0dfaa3f140cf4178dbfe316b"
server_key_version: "encryptionkey@0"
I20260806 22:54:09.191793 20357 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786056586788339-29547-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786056586788339-29547-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786056586788339-29547-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:54:09.202266 20357 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:54:09.204511 20357 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:54:09.204664 20357 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:54:09.208786 20357 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.254:36617
I20260806 22:54:09.208819 20426 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.254:36617 every 8 connection(s)
I20260806 22:54:09.209132 20357 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
I20260806 22:54:09.209961 20427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:54:09.212301 20427 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f1e0f110f8a47c187c9fe4708a3c5ec: Bootstrap starting.
I20260806 22:54:09.212894 20427 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9f1e0f110f8a47c187c9fe4708a3c5ec: Neither blocks nor log segments found. Creating new log.
I20260806 22:54:09.213037 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 20357
I20260806 22:54:09.213178 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786056586788339-29547-0/minicluster-data/master-0/wal/instance
I20260806 22:54:09.213245 20427 log.cc:824] T 00000000000000000000000000000000 P 9f1e0f110f8a47c187c9fe4708a3c5ec: Log is configured to *not* fsync() on all Append() calls
I20260806 22:54:09.213394 29547 external_mini_cluster.cc:1428] Setting key ccef6e0cbfbba5ecd1d0fe1914f6bcee
I20260806 22:54:09.214099 20427 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f1e0f110f8a47c187c9fe4708a3c5ec: No bootstrap required, opened a new log
I20260806 22:54:09.215847 20427 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9f1e0f110f8a47c187c9fe4708a3c5ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f1e0f110f8a47c187c9fe4708a3c5ec" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 36617 } }
I20260806 22:54:09.216007 20427 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9f1e0f110f8a47c187c9fe4708a3c5ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:54:09.216046 20427 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9f1e0f110f8a47c187c9fe4708a3c5ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f1e0f110f8a47c187c9fe4708a3c5ec, State: Initialized, Role: FOLLOWER
I20260806 22:54:09.216177 20427 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9f1e0f110f8a47c187c9fe4708a3c5ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f1e0f110f8a47c187c9fe4708a3c5ec" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 36617 } }
I20260806 22:54:09.216274 20427 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9f1e0f110f8a47c187c9fe4708a3c5ec [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:54:09.216332 20427 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9f1e0f110f8a47c187c9fe4708a3c5ec [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:54:09.216382 20427 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9f1e0f110f8a47c187c9fe4708a3c5ec [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:54:09.216919 20427 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9f1e0f110f8a47c187c9fe4708a3c5ec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f1e0f110f8a47c187c9fe4708a3c5ec" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 36617 } }
I20260806 22:54:09.217010 20427 leader_election.cc:302] T 00000000000000000000000000000000 P 9f1e0f110f8a47c187c9fe4708a3c5ec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9f1e0f110f8a47c187c9fe4708a3c5ec; no voters:
I20260806 22:54:09.217207 20427 leader_election.cc:288] T 00000000000000000000000000000000 P 9f1e0f110f8a47c187c9fe4708a3c5ec [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:54:09.217231 20433 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9f1e0f110f8a47c187c9fe4708a3c5ec [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:54:09.217453 20433 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9f1e0f110f8a47c187c9fe4708a3c5ec [term 1 LEADER]: Becoming Leader. State: Replica: 9f1e0f110f8a47c187c9fe4708a3c5ec, State: Running, Role: LEADER
I20260806 22:54:09.217474 20427 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9f1e0f110f8a47c187c9fe4708a3c5ec [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:54:09.217583 20433 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9f1e0f110f8a47c187c9fe4708a3c5ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f1e0f110f8a47c187c9fe4708a3c5ec" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 36617 } }
I20260806 22:54:09.217952 20433 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f1e0f110f8a47c187c9fe4708a3c5ec [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9f1e0f110f8a47c187c9fe4708a3c5ec. Latest consensus state: current_term: 1 leader_uuid: "9f1e0f110f8a47c187c9fe4708a3c5ec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f1e0f110f8a47c187c9fe4708a3c5ec" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 36617 } } }
I20260806 22:54:09.217965 20434 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f1e0f110f8a47c187c9fe4708a3c5ec [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9f1e0f110f8a47c187c9fe4708a3c5ec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f1e0f110f8a47c187c9fe4708a3c5ec" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 36617 } } }
I20260806 22:54:09.218044 20433 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f1e0f110f8a47c187c9fe4708a3c5ec [sys.catalog]: This master's current role is: LEADER
I20260806 22:54:09.218070 20434 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f1e0f110f8a47c187c9fe4708a3c5ec [sys.catalog]: This master's current role is: LEADER
I20260806 22:54:09.218254 20442 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:54:09.218739 20442 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:54:09.220628 20442 catalog_manager.cc:1347] Generated new cluster ID: faefc88f6805463786afce3ffec84df0
I20260806 22:54:09.220683 20442 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:54:09.221158 20432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:54:09.214024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47647 (local address 127.28.218.254:36617)
0806 22:54:09.214269 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0806 22:54:09.214282 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 22:54:09.214308 (+ 26us) server_negotiation.cc:407] Connection header received
0806 22:54:09.215832 (+ 1524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:54:09.215839 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:54:09.215898 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:54:09.216034 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:54:09.216255 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:54:09.217788 (+ 1533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:54:09.218946 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:54:09.219614 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:54:09.219757 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:54:09.220453 (+ 696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:54:09.220457 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:54:09.220457 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:54:09.220691 (+ 234us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:54:09.220742 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:54:09.220755 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:54:09.220811 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 22:54:09.220913 (+ 102us) server_negotiation.cc:299] Negotiation successful
0806 22:54:09.220981 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":115,"threads_started":1}
I20260806 22:54:09.221971 29547 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:54:09.225821 20432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:54:09.223295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48185 (local address 127.28.218.254:36617)
0806 22:54:09.223349 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0806 22:54:09.223351 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:54:09.223360 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:54:09.223399 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:54:09.223399 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:54:09.223408 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:54:09.223453 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:54:09.223588 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:54:09.224027 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:54:09.224712 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:54:09.224905 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:54:09.225002 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:54:09.225582 (+ 580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:54:09.225584 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:54:09.225584 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:54:09.225588 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:54:09.225599 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:54:09.225603 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:54:09.225645 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0806 22:54:09.225730 (+ 85us) server_negotiation.cc:299] Negotiation successful
0806 22:54:09.225749 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 22:54:09.258613 20442 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:54:09.259107 20442 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:54:09.263908 20442 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9f1e0f110f8a47c187c9fe4708a3c5ec: Generated new TSK 0
I20260806 22:54:09.264083 20442 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:54:09.269209 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 20357
2026-08-06T22:54:09Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1341 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786056586788339-29547-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:54:09 dist-test-slave-1xrh krb5kdc[20460](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:54:09 dist-test-slave-1xrh krb5kdc[20460](info): set up 2 sockets
Aug 06 22:54:09 dist-test-slave-1xrh krb5kdc[20460](info): commencing operation
krb5kdc: starting...
W20260806 22:54:10.319311 29547 mini_kdc.cc:121] Time spent starting KDC: real 1.031s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786056586788339-29547-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786056586788339-29547-0/krb5kdc/test-user.keytab.
Aug 06 22:54:10 dist-test-slave-1xrh krb5kdc[20460](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056850, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:54:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:54:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.218.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.218.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.218.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab.
Entry for principal kudu/127.28.218.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab.
I20260806 22:54:10.438160 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786056586788339-29547-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786056586788339-29547-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786056586788339-29547-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.218.254:40093
--webserver_interface=127.28.218.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab
--principal=kudu/127.28.218.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.218.212:43583
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.218.254:40093
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:54:10.518302 20474 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:54:10.518527 20474 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:54:10.518567 20474 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:54:10.519767 20474 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:54:10.519845 20474 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:54:10.519867 20474 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:54:10.519892 20474 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:54:10.519920 20474 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:54:10.521365 20474 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:43583
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786056586788339-29547-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786056586788339-29547-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.218.254:40093
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab
--principal=kudu/127.28.218.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.218.254:40093
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.218.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20474
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786056586788339-29547-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:54:10.521625 20474 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:54:10.521878 20474 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:54:10.526727 20481 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:54:10.526724 20480 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:54:10.526726 20483 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:54:10.526743 20474 server_base.cc:1034] running on GCE node
I20260806 22:54:10.527235 20474 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:54:10.527534 20474 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:54:10.528698 20474 hybrid_clock.cc:648] HybridClock initialized: now 1786056850528684 us; error 45 us; skew 500 ppm
Aug 06 22:54:10 dist-test-slave-1xrh krb5kdc[20460](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056850, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:54:10.531206 20474 init.cc:377] Logged in from keytab as kudu/127.28.218.254@KRBTEST.COM (short username kudu)
I20260806 22:54:10.531625 20474 webserver.cc:460] Webserver started at http://127.28.218.254:35299/ using document root <none> and password file <none>
I20260806 22:54:10.531849 20474 fs_manager.cc:359] Metadata directory not provided
I20260806 22:54:10.531914 20474 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:54:10.532064 20474 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:54:10.533057 20474 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786056586788339-29547-0/minicluster-data/master-0/data/instance:
uuid: "a95bacb1e2ce4999b71a546d9dfb6439"
format_stamp: "Formatted at 2026-08-06 22:54:10 on dist-test-slave-1xrh"
server_key: "7669b8a0112e37fca493b5c0ceff05d2"
server_key_iv: "6ba30f4495692eec05e3d8efdf27afb0"
server_key_version: "encryptionkey@0"
I20260806 22:54:10.533394 20474 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786056586788339-29547-0/minicluster-data/master-0/wal/instance:
uuid: "a95bacb1e2ce4999b71a546d9dfb6439"
format_stamp: "Formatted at 2026-08-06 22:54:10 on dist-test-slave-1xrh"
server_key: "7669b8a0112e37fca493b5c0ceff05d2"
server_key_iv: "6ba30f4495692eec05e3d8efdf27afb0"
server_key_version: "encryptionkey@0"
I20260806 22:54:10.534662 20474 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 22:54:10.535466 20490 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:54:10.535636 20474 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:54:10.535732 20474 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786056586788339-29547-0/minicluster-data/master-0/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786056586788339-29547-0/minicluster-data/master-0/wal
uuid: "a95bacb1e2ce4999b71a546d9dfb6439"
format_stamp: "Formatted at 2026-08-06 22:54:10 on dist-test-slave-1xrh"
server_key: "7669b8a0112e37fca493b5c0ceff05d2"
server_key_iv: "6ba30f4495692eec05e3d8efdf27afb0"
server_key_version: "encryptionkey@0"
I20260806 22:54:10.535812 20474 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786056586788339-29547-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786056586788339-29547-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786056586788339-29547-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:54:10.538930 20474 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:54:10.540704 20474 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:54:10.540855 20474 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:54:10.545061 20474 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.254:40093
I20260806 22:54:10.545092 20545 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.254:40093 every 8 connection(s)
I20260806 22:54:10.545380 20474 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
I20260806 22:54:10.546271 20546 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:54:10.548382 20546 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a95bacb1e2ce4999b71a546d9dfb6439: Bootstrap starting.
I20260806 22:54:10.548862 20546 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a95bacb1e2ce4999b71a546d9dfb6439: Neither blocks nor log segments found. Creating new log.
I20260806 22:54:10.549103 20546 log.cc:824] T 00000000000000000000000000000000 P a95bacb1e2ce4999b71a546d9dfb6439: Log is configured to *not* fsync() on all Append() calls
I20260806 22:54:10.549714 20546 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a95bacb1e2ce4999b71a546d9dfb6439: No bootstrap required, opened a new log
I20260806 22:54:10.551182 20546 raft_consensus.cc:348] T 00000000000000000000000000000000 P a95bacb1e2ce4999b71a546d9dfb6439 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a95bacb1e2ce4999b71a546d9dfb6439" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 40093 } }
I20260806 22:54:10.551338 20546 raft_consensus.cc:374] T 00000000000000000000000000000000 P a95bacb1e2ce4999b71a546d9dfb6439 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:54:10.551360 20546 raft_consensus.cc:729] T 00000000000000000000000000000000 P a95bacb1e2ce4999b71a546d9dfb6439 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a95bacb1e2ce4999b71a546d9dfb6439, State: Initialized, Role: FOLLOWER
I20260806 22:54:10.551452 20546 consensus_queue.cc:260] T 00000000000000000000000000000000 P a95bacb1e2ce4999b71a546d9dfb6439 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a95bacb1e2ce4999b71a546d9dfb6439" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 40093 } }
I20260806 22:54:10.551501 20546 raft_consensus.cc:388] T 00000000000000000000000000000000 P a95bacb1e2ce4999b71a546d9dfb6439 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:54:10.551520 20546 raft_consensus.cc:482] T 00000000000000000000000000000000 P a95bacb1e2ce4999b71a546d9dfb6439 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:54:10.551545 20546 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a95bacb1e2ce4999b71a546d9dfb6439 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:54:10.552259 20546 raft_consensus.cc:504] T 00000000000000000000000000000000 P a95bacb1e2ce4999b71a546d9dfb6439 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a95bacb1e2ce4999b71a546d9dfb6439" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 40093 } }
I20260806 22:54:10.552374 20546 leader_election.cc:302] T 00000000000000000000000000000000 P a95bacb1e2ce4999b71a546d9dfb6439 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a95bacb1e2ce4999b71a546d9dfb6439; no voters:
I20260806 22:54:10.552538 20546 leader_election.cc:288] T 00000000000000000000000000000000 P a95bacb1e2ce4999b71a546d9dfb6439 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:54:10.552556 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 20474
I20260806 22:54:10.552562 20549 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a95bacb1e2ce4999b71a546d9dfb6439 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:54:10.552657 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786056586788339-29547-0/minicluster-data/master-0/wal/instance
I20260806 22:54:10.552742 20549 raft_consensus.cc:686] T 00000000000000000000000000000000 P a95bacb1e2ce4999b71a546d9dfb6439 [term 1 LEADER]: Becoming Leader. State: Replica: a95bacb1e2ce4999b71a546d9dfb6439, State: Running, Role: LEADER
I20260806 22:54:10.552786 20546 sys_catalog.cc:564] T 00000000000000000000000000000000 P a95bacb1e2ce4999b71a546d9dfb6439 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:54:10.552877 29547 external_mini_cluster.cc:1428] Setting key 5c43928a3b041dd68eb99feae4d52ff8
I20260806 22:54:10.552855 20549 consensus_queue.cc:237] T 00000000000000000000000000000000 P a95bacb1e2ce4999b71a546d9dfb6439 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a95bacb1e2ce4999b71a546d9dfb6439" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 40093 } }
I20260806 22:54:10.553192 20550 sys_catalog.cc:455] T 00000000000000000000000000000000 P a95bacb1e2ce4999b71a546d9dfb6439 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a95bacb1e2ce4999b71a546d9dfb6439. Latest consensus state: current_term: 1 leader_uuid: "a95bacb1e2ce4999b71a546d9dfb6439" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a95bacb1e2ce4999b71a546d9dfb6439" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 40093 } } }
I20260806 22:54:10.553326 20550 sys_catalog.cc:458] T 00000000000000000000000000000000 P a95bacb1e2ce4999b71a546d9dfb6439 [sys.catalog]: This master's current role is: LEADER
I20260806 22:54:10.553453 20551 sys_catalog.cc:455] T 00000000000000000000000000000000 P a95bacb1e2ce4999b71a546d9dfb6439 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a95bacb1e2ce4999b71a546d9dfb6439" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a95bacb1e2ce4999b71a546d9dfb6439" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 40093 } } }
I20260806 22:54:10.553566 20551 sys_catalog.cc:458] T 00000000000000000000000000000000 P a95bacb1e2ce4999b71a546d9dfb6439 [sys.catalog]: This master's current role is: LEADER
I20260806 22:54:10.553597 20559 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:54:10.554086 20559 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:54:10.556241 20559 catalog_manager.cc:1347] Generated new cluster ID: 0802315aaeed4d7cb0092c9cef2d255a
I20260806 22:54:10.556285 20557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:54:10.553331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54121 (local address 127.28.218.254:40093)
0806 22:54:10.554088 (+ 757us) server_negotiation.cc:206] Beginning negotiation
0806 22:54:10.554110 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0806 22:54:10.554719 (+ 609us) server_negotiation.cc:407] Connection header received
0806 22:54:10.554857 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:54:10.554862 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:54:10.554882 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:54:10.555009 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:54:10.555827 (+ 818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:54:10.555831 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:54:10.555831 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:54:10.555836 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:54:10.555882 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:54:10.555896 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:54:10.555958 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 22:54:10.556079 (+ 121us) server_negotiation.cc:299] Negotiation successful
0806 22:54:10.556118 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":704,"thread_start_us":72,"threads_started":1}
I20260806 22:54:10.556293 20559 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:54:10.557274 29547 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:54:10.559691 20557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:54:10.558707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36489 (local address 127.28.218.254:40093)
0806 22:54:10.558738 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0806 22:54:10.558740 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:54:10.558836 (+ 96us) server_negotiation.cc:407] Connection header received
0806 22:54:10.558881 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:54:10.558882 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:54:10.558896 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:54:10.558947 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:54:10.559481 (+ 534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:54:10.559482 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:54:10.559483 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:54:10.559487 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:54:10.559501 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:54:10.559505 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:54:10.559567 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 22:54:10.559623 (+ 56us) server_negotiation.cc:299] Negotiation successful
0806 22:54:10.559634 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 22:54:10.577798 20559 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:54:10.577952 20559 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:54:10.586061 20559 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a95bacb1e2ce4999b71a546d9dfb6439: Generated new TSK 0
I20260806 22:54:10.586238 20559 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:54:10.589277 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 20474
2026-08-06T22:54:10Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1320 ms)
[----------] 18 tests from AuthTokenIssuingTest (24042 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056586788339-29547-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:54:10 dist-test-slave-1xrh krb5kdc[20578](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:54:10 dist-test-slave-1xrh krb5kdc[20578](info): set up 2 sockets
Aug 06 22:54:10 dist-test-slave-1xrh krb5kdc[20578](info): commencing operation
krb5kdc: starting...
W20260806 22:54:11.635037 29547 mini_kdc.cc:121] Time spent starting KDC: real 1.026s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056586788339-29547-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056586788339-29547-0/krb5kdc/test-user.keytab.
Aug 06 22:54:11 dist-test-slave-1xrh krb5kdc[20578](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056851, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:54:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:54:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.218.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.218.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.218.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab.
Entry for principal kudu/127.28.218.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab.
I20260806 22:54:11.754302 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056586788339-29547-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056586788339-29547-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056586788339-29547-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.218.254:43403
--webserver_interface=127.28.218.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab
--principal=kudu/127.28.218.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.218.212:44693
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.218.254:43403,127.28.218.253:36185,127.28.218.252:41547
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:54:11.836531 20591 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 22:54:11.836723 20591 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:54:11.836740 20591 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:54:11.836759 20591 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:54:11.837996 20591 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:54:11.838034 20591 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:54:11.838048 20591 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:54:11.838061 20591 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:54:11.838110 20591 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:54:11.839632 20591 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:44693
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056586788339-29547-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056586788339-29547-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.218.254:43403,127.28.218.253:36185,127.28.218.252:41547
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab
--principal=kudu/127.28.218.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.218.254:43403
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.218.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20591
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056586788339-29547-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:54:11.839874 20591 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:54:11.840093 20591 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:54:11.844707 20597 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:54:11.844719 20598 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:54:11.844723 20600 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:54:11.845080 20591 server_base.cc:1034] running on GCE node
I20260806 22:54:11.845319 20591 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:54:11.845578 20591 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:54:11.846716 20591 hybrid_clock.cc:648] HybridClock initialized: now 1786056851846691 us; error 38 us; skew 500 ppm
Aug 06 22:54:11 dist-test-slave-1xrh krb5kdc[20578](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056851, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:54:11.849022 20591 init.cc:377] Logged in from keytab as kudu/127.28.218.254@KRBTEST.COM (short username kudu)
I20260806 22:54:11.849408 20591 webserver.cc:460] Webserver started at http://127.28.218.254:38789/ using document root <none> and password file <none>
I20260806 22:54:11.849617 20591 fs_manager.cc:359] Metadata directory not provided
I20260806 22:54:11.849682 20591 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:54:11.849828 20591 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:54:11.850785 20591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056586788339-29547-0/minicluster-data/master-0/data/instance:
uuid: "67499e7f32e249ca834e2839ad85f46e"
format_stamp: "Formatted at 2026-08-06 22:54:11 on dist-test-slave-1xrh"
server_key: "06bf4283ce1884122a689bd29f45ea03"
server_key_iv: "0ef513860f71934244e9655a1d6a465f"
server_key_version: "encryptionkey@0"
I20260806 22:54:11.851174 20591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056586788339-29547-0/minicluster-data/master-0/wal/instance:
uuid: "67499e7f32e249ca834e2839ad85f46e"
format_stamp: "Formatted at 2026-08-06 22:54:11 on dist-test-slave-1xrh"
server_key: "06bf4283ce1884122a689bd29f45ea03"
server_key_iv: "0ef513860f71934244e9655a1d6a465f"
server_key_version: "encryptionkey@0"
I20260806 22:54:11.852463 20591 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:54:11.853214 20609 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:54:11.853375 20591 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:54:11.853473 20591 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056586788339-29547-0/minicluster-data/master-0/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056586788339-29547-0/minicluster-data/master-0/wal
uuid: "67499e7f32e249ca834e2839ad85f46e"
format_stamp: "Formatted at 2026-08-06 22:54:11 on dist-test-slave-1xrh"
server_key: "06bf4283ce1884122a689bd29f45ea03"
server_key_iv: "0ef513860f71934244e9655a1d6a465f"
server_key_version: "encryptionkey@0"
I20260806 22:54:11.853561 20591 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056586788339-29547-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056586788339-29547-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056586788339-29547-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:54:11.874065 20591 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:54:11.876003 20591 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:54:11.876148 20591 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:54:11.880029 20591 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.254:43403
I20260806 22:54:11.880095 20665 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.254:43403 every 8 connection(s)
I20260806 22:54:11.880445 20591 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
I20260806 22:54:11.881301 20666 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:54:11.882745 20666 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.218.254" port: 43403 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 22:54:11 dist-test-slave-1xrh krb5kdc[20578](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056851, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.254@KRBTEST.COM for kudu/127.28.218.254@KRBTEST.COM
I20260806 22:54:11.888720 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 20591
I20260806 22:54:11.888841 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056586788339-29547-0/minicluster-data/master-0/wal/instance
I20260806 22:54:11.889060 29547 external_mini_cluster.cc:1428] Setting key 2c9568a9e432ae380042b1f8b56fc029
I20260806 22:54:11.892663 20668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:54:11.883373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:43403 (local address 127.0.0.1:54378)
0806 22:54:11.883515 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0806 22:54:11.883549 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0806 22:54:11.884045 (+ 496us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:54:11.884296 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:54:11.884299 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:54:11.884528 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:54:11.884674 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:54:11.884677 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:54:11.885663 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:54:11.885664 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:54:11.886326 (+ 662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:54:11.886328 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:54:11.886812 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:54:11.886813 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:54:11.886875 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:54:11.887437 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:54:11.887445 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:54:11.888937 (+ 1492us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:54:11.891023 (+ 2086us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:54:11.891031 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:54:11.891034 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:54:11.891383 (+ 349us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:54:11.891616 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:54:11.891617 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:54:11.891617 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:54:11.891743 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:54:11.892151 (+ 408us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:54:11.892158 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:54:11.892231 (+ 73us) client_negotiation.cc:769] Sending connection context
0806 22:54:11.892329 (+ 98us) client_negotiation.cc:240] Negotiation successful
0806 22:54:11.892489 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":63,"threads_started":1}
I20260806 22:54:11.892760 20669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:54:11.883515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54378 (local address 127.28.218.254:43403)
0806 22:54:11.883639 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0806 22:54:11.883642 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:54:11.883669 (+ 27us) server_negotiation.cc:407] Connection header received
0806 22:54:11.884179 (+ 510us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:54:11.884180 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:54:11.884196 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:54:11.884272 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:54:11.884754 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:54:11.885580 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:54:11.886425 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:54:11.886731 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:54:11.886816 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:54:11.889024 (+ 2208us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:54:11.889034 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:54:11.889034 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:54:11.889041 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:54:11.890902 (+ 1861us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:54:11.891467 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:54:11.891469 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:54:11.891470 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:54:11.891545 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:54:11.891855 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:54:11.891856 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:54:11.891856 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:54:11.892054 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:54:11.892143 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 22:54:11.892388 (+ 245us) server_negotiation.cc:299] Negotiation successful
0806 22:54:11.892563 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":64,"server-negotiator.queue_time_us":89,"thread_start_us":60,"threads_started":1}
I20260806 22:54:11.893426 20666 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.218.253" port: 36185 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:54:11.893744 20668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:54:11.893632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.253:36185 (local address 127.0.0.1:43566)
0806 22:54:11.893678 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0806 22:54:11.893719 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.253:36185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260806 22:54:11.893877 20611 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.28.218.253:36185: connect: Connection refused (error 111)
I20260806 22:54:11.894284 20668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:54:11.894179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.253:36185 (local address 127.0.0.1:43568)
0806 22:54:11.894226 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0806 22:54:11.894249 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.253:36185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260806 22:54:11.894383 20666 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.218.253:36185: Network error: Client connection negotiation failed: client connection to 127.28.218.253:36185: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.28.218.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.218.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.218.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.253.keytab.
Entry for principal kudu/127.28.218.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.253.keytab.
I20260806 22:54:11.914269 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056586788339-29547-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056586788339-29547-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056586788339-29547-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056586788339-29547-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.218.253:36185
--webserver_interface=127.28.218.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.253.keytab
--principal=kudu/127.28.218.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.218.212:44693
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.218.254:43403,127.28.218.253:36185,127.28.218.252:41547
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:54:11.943607 20666 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.218.253" port: 36185 } attempt: 1
I20260806 22:54:11.944115 20668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:54:11.943979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.253:36185 (local address 127.0.0.1:43584)
0806 22:54:11.944044 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0806 22:54:11.944083 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.253:36185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260806 22:54:11.944434 20668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:54:11.944333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.253:36185 (local address 127.0.0.1:43594)
0806 22:54:11.944380 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0806 22:54:11.944405 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.253:36185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260806 22:54:11.944521 20666 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.218.253:36185: Network error: Client connection negotiation failed: client connection to 127.28.218.253:36185: connect: Connection refused (error 111)
W20260806 22:54:11.993309 20673 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 22:54:11.993538 20673 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:54:11.993569 20673 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:54:11.993613 20673 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:54:11.994758 20673 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:54:11.994823 20673 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:54:11.994846 20673 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:54:11.994874 20673 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:54:11.994904 20673 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:54:11.996361 20673 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:44693
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056586788339-29547-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056586788339-29547-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.28.218.254:43403,127.28.218.253:36185,127.28.218.252:41547
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.253.keytab
--principal=kudu/127.28.218.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.218.253:36185
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056586788339-29547-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.28.218.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20673
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056586788339-29547-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:54:11.996634 20673 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:54:11.996860 20673 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:54:12.001382 20681 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:54:12.001411 20678 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:54:12.001534 20673 server_base.cc:1034] running on GCE node
W20260806 22:54:12.001382 20679 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:54:12.001893 20673 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:54:12.002210 20673 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:54:12.003373 20673 hybrid_clock.cc:648] HybridClock initialized: now 1786056852003374 us; error 48 us; skew 500 ppm
Aug 06 22:54:12 dist-test-slave-1xrh krb5kdc[20578](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056852, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:54:12.005788 20673 init.cc:377] Logged in from keytab as kudu/127.28.218.253@KRBTEST.COM (short username kudu)
I20260806 22:54:12.006279 20673 webserver.cc:460] Webserver started at http://127.28.218.253:42397/ using document root <none> and password file <none>
I20260806 22:54:12.006493 20673 fs_manager.cc:359] Metadata directory not provided
I20260806 22:54:12.006558 20673 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:54:12.006711 20673 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:54:12.007759 20673 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056586788339-29547-0/minicluster-data/master-1/data/instance:
uuid: "a5a8a2ee77ca4812b90172945f44652e"
format_stamp: "Formatted at 2026-08-06 22:54:12 on dist-test-slave-1xrh"
server_key: "46e072ef67d31f67a1654deed09ab5ce"
server_key_iv: "8c5097699a1f9cff00a8756b94892b54"
server_key_version: "encryptionkey@0"
I20260806 22:54:12.008107 20673 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056586788339-29547-0/minicluster-data/master-1/wal/instance:
uuid: "a5a8a2ee77ca4812b90172945f44652e"
format_stamp: "Formatted at 2026-08-06 22:54:12 on dist-test-slave-1xrh"
server_key: "46e072ef67d31f67a1654deed09ab5ce"
server_key_iv: "8c5097699a1f9cff00a8756b94892b54"
server_key_version: "encryptionkey@0"
I20260806 22:54:12.009472 20673 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:54:12.010221 20688 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:54:12.010387 20673 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:54:12.010485 20673 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056586788339-29547-0/minicluster-data/master-1/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056586788339-29547-0/minicluster-data/master-1/wal
uuid: "a5a8a2ee77ca4812b90172945f44652e"
format_stamp: "Formatted at 2026-08-06 22:54:12 on dist-test-slave-1xrh"
server_key: "46e072ef67d31f67a1654deed09ab5ce"
server_key_iv: "8c5097699a1f9cff00a8756b94892b54"
server_key_version: "encryptionkey@0"
I20260806 22:54:12.010563 20673 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056586788339-29547-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056586788339-29547-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056586788339-29547-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:54:12.012658 20666 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.218.253" port: 36185 } attempt: 2
I20260806 22:54:12.013036 20668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:54:12.012913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.253:36185 (local address 127.0.0.1:43604)
0806 22:54:12.012972 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0806 22:54:12.013007 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.253:36185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260806 22:54:12.013362 20668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:54:12.013264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.253:36185 (local address 127.0.0.1:43606)
0806 22:54:12.013314 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0806 22:54:12.013344 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.253:36185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260806 22:54:12.013449 20666 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.218.253:36185: Network error: Client connection negotiation failed: client connection to 127.28.218.253:36185: connect: Connection refused (error 111)
I20260806 22:54:12.019984 20673 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:54:12.021713 20673 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:54:12.021852 20673 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:54:12.025923 20673 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.253:36185
I20260806 22:54:12.025954 20744 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.253:36185 every 8 connection(s)
I20260806 22:54:12.026276 20673 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056586788339-29547-0/minicluster-data/master-1/data/info.pb
I20260806 22:54:12.027076 20745 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:54:12.028748 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 20673
I20260806 22:54:12.028543 20745 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.218.254" port: 43403 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:54:12.028873 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056586788339-29547-0/minicluster-data/master-1/wal/instance
I20260806 22:54:12.029059 29547 external_mini_cluster.cc:1428] Setting key 6cca58c54df9354d8b4f67c4fab09fe4
Aug 06 22:54:12 dist-test-slave-1xrh krb5kdc[20578](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056852, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.253@KRBTEST.COM for kudu/127.28.218.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.28.218.252@KRBTEST.COM; defaulting to no policy
I20260806 22:54:12.038589 20669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:54:12.029408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54390 (local address 127.28.218.254:43403)
0806 22:54:12.029471 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0806 22:54:12.029473 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:54:12.029632 (+ 159us) server_negotiation.cc:407] Connection header received
0806 22:54:12.030053 (+ 421us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:54:12.030055 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:54:12.030069 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:54:12.030134 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:54:12.030652 (+ 518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:54:12.031298 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:54:12.032656 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:54:12.032882 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:54:12.032932 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:54:12.035750 (+ 2818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:54:12.035752 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:54:12.035752 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:54:12.035758 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:54:12.037164 (+ 1406us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:54:12.037641 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:54:12.037641 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:54:12.037642 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:54:12.037689 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:54:12.037973 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:54:12.037974 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:54:12.037974 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:54:12.038068 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:54:12.038116 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 22:54:12.038464 (+ 348us) server_negotiation.cc:299] Negotiation successful
0806 22:54:12.038529 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 22:54:12.038857 20747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:54:12.029323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:43403 (local address 127.0.0.1:54390)
0806 22:54:12.029506 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0806 22:54:12.029544 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 22:54:12.029967 (+ 423us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:54:12.030178 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:54:12.030181 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:54:12.030395 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:54:12.030576 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:54:12.030579 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:54:12.031388 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:54:12.031389 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:54:12.032570 (+ 1181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:54:12.032573 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:54:12.032964 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:54:12.032965 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:54:12.033126 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:54:12.033650 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:54:12.033662 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:54:12.035656 (+ 1994us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:54:12.037252 (+ 1596us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:54:12.037259 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:54:12.037261 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:54:12.037563 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:54:12.037756 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:54:12.037757 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:54:12.037758 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:54:12.037910 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:54:12.038134 (+ 224us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:54:12.038141 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:54:12.038325 (+ 184us) client_negotiation.cc:769] Sending connection context
0806 22:54:12.038448 (+ 123us) client_negotiation.cc:240] Negotiation successful
0806 22:54:12.038606 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":82,"threads_started":1}
I20260806 22:54:12.039505 20745 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.218.253" port: 36185 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.218.252@KRBTEST.COM" created.
Aug 06 22:54:12 dist-test-slave-1xrh krb5kdc[20578](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056852, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.253@KRBTEST.COM for kudu/127.28.218.253@KRBTEST.COM
I20260806 22:54:12.046825 20747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:54:12.039719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.253:36185 (local address 127.0.0.1:43616)
0806 22:54:12.039766 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0806 22:54:12.039777 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:54:12.039860 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:54:12.040047 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:54:12.040047 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:54:12.040153 (+ 106us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:54:12.040201 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:54:12.040201 (+ 0us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:54:12.041098 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:54:12.041099 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:54:12.041672 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:54:12.041674 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:54:12.042153 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:54:12.042154 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:54:12.042217 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:54:12.042622 (+ 405us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:54:12.042627 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:54:12.044091 (+ 1464us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:54:12.045773 (+ 1682us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:54:12.045774 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:54:12.045774 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:54:12.045989 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:54:12.046184 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:54:12.046185 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:54:12.046185 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:54:12.046250 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:54:12.046494 (+ 244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:54:12.046495 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:54:12.046539 (+ 44us) client_negotiation.cc:769] Sending connection context
0806 22:54:12.046618 (+ 79us) client_negotiation.cc:240] Negotiation successful
0806 22:54:12.046711 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260806 22:54:12.046844 20750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:54:12.039790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43616 (local address 127.28.218.253:36185)
0806 22:54:12.039922 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0806 22:54:12.039926 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:54:12.039951 (+ 25us) server_negotiation.cc:407] Connection header received
0806 22:54:12.039978 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:54:12.039979 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:54:12.039989 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:54:12.040046 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:54:12.040285 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:54:12.040984 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:54:12.041768 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:54:12.042068 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:54:12.042153 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:54:12.044172 (+ 2019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:54:12.044178 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:54:12.044179 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:54:12.044184 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:54:12.045674 (+ 1490us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:54:12.046074 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:54:12.046076 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:54:12.046076 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:54:12.046122 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:54:12.046310 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:54:12.046311 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:54:12.046311 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:54:12.046419 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:54:12.046484 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 22:54:12.046637 (+ 153us) server_negotiation.cc:299] Negotiation successful
0806 22:54:12.046753 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":60,"server-negotiator.queue_time_us":107,"thread_start_us":80,"threads_started":1}
I20260806 22:54:12.047474 20745 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.218.252" port: 41547 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:54:12.047802 20747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:54:12.047675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.252:41547 (local address 127.0.0.1:47116)
0806 22:54:12.047727 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0806 22:54:12.047778 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.252:41547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260806 22:54:12.047950 20692 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.28.218.252:41547: connect: Connection refused (error 111)
I20260806 22:54:12.048410 20747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:54:12.048292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.252:41547 (local address 127.0.0.1:47122)
0806 22:54:12.048357 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0806 22:54:12.048388 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.252:41547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260806 22:54:12.048516 20745 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.218.252:41547: Network error: Client connection negotiation failed: client connection to 127.28.218.252:41547: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.218.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.252.keytab.
Entry for principal kudu/127.28.218.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.252.keytab.
I20260806 22:54:12.058444 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056586788339-29547-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056586788339-29547-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056586788339-29547-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056586788339-29547-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.218.252:41547
--webserver_interface=127.28.218.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.252.keytab
--principal=kudu/127.28.218.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.218.212:44693
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.218.254:43403,127.28.218.253:36185,127.28.218.252:41547
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:54:12.097744 20745 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.218.252" port: 41547 } attempt: 1
I20260806 22:54:12.098248 20747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:54:12.098104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.252:41547 (local address 127.0.0.1:47126)
0806 22:54:12.098169 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0806 22:54:12.098214 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.252:41547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260806 22:54:12.098627 20747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:54:12.098516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.252:41547 (local address 127.0.0.1:47142)
0806 22:54:12.098562 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0806 22:54:12.098605 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.252:41547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260806 22:54:12.098748 20745 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.218.252:41547: Network error: Client connection negotiation failed: client connection to 127.28.218.252:41547: connect: Connection refused (error 111)
I20260806 22:54:12.104591 20666 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.218.253" port: 36185 } attempt: 3
Aug 06 22:54:12 dist-test-slave-1xrh krb5kdc[20578](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056851, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.254@KRBTEST.COM for kudu/127.28.218.253@KRBTEST.COM
I20260806 22:54:12.111697 20668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:54:12.104882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.253:36185 (local address 127.0.0.1:43628)
0806 22:54:12.104937 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0806 22:54:12.104949 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:54:12.105016 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:54:12.105171 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:54:12.105172 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:54:12.105330 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:54:12.105394 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:54:12.105396 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:54:12.106048 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:54:12.106050 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:54:12.106575 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:54:12.106577 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:54:12.106980 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:54:12.106981 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:54:12.107050 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:54:12.107611 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:54:12.107617 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:54:12.109034 (+ 1417us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:54:12.110767 (+ 1733us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:54:12.110768 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:54:12.110769 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:54:12.110917 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:54:12.111111 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:54:12.111112 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:54:12.111112 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:54:12.111222 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:54:12.111481 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:54:12.111481 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:54:12.111518 (+ 37us) client_negotiation.cc:769] Sending connection context
0806 22:54:12.111567 (+ 49us) client_negotiation.cc:240] Negotiation successful
0806 22:54:12.111638 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260806 22:54:12.111748 20750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:54:12.104941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43628 (local address 127.28.218.253:36185)
0806 22:54:12.104994 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0806 22:54:12.104996 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:54:12.105008 (+ 12us) server_negotiation.cc:407] Connection header received
0806 22:54:12.105084 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:54:12.105085 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:54:12.105097 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:54:12.105166 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:54:12.105496 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:54:12.105969 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:54:12.106663 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:54:12.106881 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:54:12.106968 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:54:12.109132 (+ 2164us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:54:12.109133 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:54:12.109134 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:54:12.109141 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:54:12.110688 (+ 1547us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:54:12.110992 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:54:12.110993 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:54:12.110993 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:54:12.111051 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:54:12.111283 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:54:12.111284 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:54:12.111284 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:54:12.111417 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:54:12.111486 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 22:54:12.111596 (+ 110us) server_negotiation.cc:299] Negotiation successful
0806 22:54:12.111668 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 22:54:12.112169 20666 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.218.252" port: 41547 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:54:12.112488 20668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:54:12.112394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.252:41547 (local address 127.0.0.1:47156)
0806 22:54:12.112437 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0806 22:54:12.112467 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.252:41547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260806 22:54:12.112782 20668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:54:12.112731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.252:41547 (local address 127.0.0.1:47170)
0806 22:54:12.112751 (+ 20us) negotiation.cc:107] Waiting for socket to connect
0806 22:54:12.112769 (+ 18us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.252:41547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":13}
W20260806 22:54:12.112864 20666 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.218.252:41547: Network error: Client connection negotiation failed: client connection to 127.28.218.252:41547: connect: Connection refused (error 111)
W20260806 22:54:12.140409 20753 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 22:54:12.140592 20753 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:54:12.140609 20753 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:54:12.140628 20753 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:54:12.141665 20753 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:54:12.141697 20753 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:54:12.141712 20753 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:54:12.141724 20753 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:54:12.141737 20753 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:54:12.143190 20753 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:44693
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056586788339-29547-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056586788339-29547-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.28.218.254:43403,127.28.218.253:36185,127.28.218.252:41547
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.252.keytab
--principal=kudu/127.28.218.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.218.252:41547
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056586788339-29547-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.28.218.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20753
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056586788339-29547-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:54:12.143375 20753 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:54:12.143527 20753 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 22:54:12.144026 20666 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.218.252" port: 41547 } attempt: 1
I20260806 22:54:12.144410 20668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:54:12.144291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.252:41547 (local address 127.0.0.1:47186)
0806 22:54:12.144345 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0806 22:54:12.144382 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.252:41547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260806 22:54:12.144838 20668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:54:12.144724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.252:41547 (local address 127.0.0.1:47202)
0806 22:54:12.144778 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0806 22:54:12.144811 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.252:41547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260806 22:54:12.144939 20666 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.218.252:41547: Network error: Client connection negotiation failed: client connection to 127.28.218.252:41547: connect: Connection refused (error 111)
W20260806 22:54:12.148160 20759 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:54:12.148168 20761 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:54:12.148207 20753 server_base.cc:1034] running on GCE node
W20260806 22:54:12.148177 20758 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:54:12.148599 20753 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:54:12.148833 20753 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:54:12.149989 20753 hybrid_clock.cc:648] HybridClock initialized: now 1786056852149986 us; error 36 us; skew 500 ppm
Aug 06 22:54:12 dist-test-slave-1xrh krb5kdc[20578](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056852, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:54:12.152375 20753 init.cc:377] Logged in from keytab as kudu/127.28.218.252@KRBTEST.COM (short username kudu)
I20260806 22:54:12.152782 20753 webserver.cc:460] Webserver started at http://127.28.218.252:42287/ using document root <none> and password file <none>
I20260806 22:54:12.152989 20753 fs_manager.cc:359] Metadata directory not provided
I20260806 22:54:12.153052 20753 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:54:12.153210 20753 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:54:12.154206 20753 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056586788339-29547-0/minicluster-data/master-2/data/instance:
uuid: "7aa720c12c624118918aff51be3d9a9f"
format_stamp: "Formatted at 2026-08-06 22:54:12 on dist-test-slave-1xrh"
server_key: "1ebf84f15f1654ebe95d4d396a2d9048"
server_key_iv: "465f57116ea2521bff38fc551daf7c11"
server_key_version: "encryptionkey@0"
I20260806 22:54:12.154558 20753 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056586788339-29547-0/minicluster-data/master-2/wal/instance:
uuid: "7aa720c12c624118918aff51be3d9a9f"
format_stamp: "Formatted at 2026-08-06 22:54:12 on dist-test-slave-1xrh"
server_key: "1ebf84f15f1654ebe95d4d396a2d9048"
server_key_iv: "465f57116ea2521bff38fc551daf7c11"
server_key_version: "encryptionkey@0"
I20260806 22:54:12.155984 20753 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:54:12.156672 20769 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:54:12.156870 20753 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:54:12.156972 20753 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056586788339-29547-0/minicluster-data/master-2/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056586788339-29547-0/minicluster-data/master-2/wal
uuid: "7aa720c12c624118918aff51be3d9a9f"
format_stamp: "Formatted at 2026-08-06 22:54:12 on dist-test-slave-1xrh"
server_key: "1ebf84f15f1654ebe95d4d396a2d9048"
server_key_iv: "465f57116ea2521bff38fc551daf7c11"
server_key_version: "encryptionkey@0"
I20260806 22:54:12.157039 20753 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056586788339-29547-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056586788339-29547-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056586788339-29547-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:54:12.166901 20745 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.218.252" port: 41547 } attempt: 2
I20260806 22:54:12.167315 20747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:54:12.167184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.252:41547 (local address 127.0.0.1:47208)
0806 22:54:12.167245 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0806 22:54:12.167281 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.252:41547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260806 22:54:12.167658 20747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:54:12.167578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.252:41547 (local address 127.0.0.1:47220)
0806 22:54:12.167619 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0806 22:54:12.167642 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.252:41547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260806 22:54:12.167773 20745 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.218.252:41547: Network error: Client connection negotiation failed: client connection to 127.28.218.252:41547: connect: Connection refused (error 111)
I20260806 22:54:12.174999 20753 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:54:12.176839 20753 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:54:12.176997 20753 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:54:12.180918 20753 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.252:41547
I20260806 22:54:12.180949 20825 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.252:41547 every 8 connection(s)
I20260806 22:54:12.181279 20753 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056586788339-29547-0/minicluster-data/master-2/data/info.pb
I20260806 22:54:12.182097 20826 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:54:12.182904 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 20753
I20260806 22:54:12.183017 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056586788339-29547-0/minicluster-data/master-2/wal/instance
I20260806 22:54:12.183269 29547 external_mini_cluster.cc:1428] Setting key 3495aedb753c7ec1c37767134007ba62
I20260806 22:54:12.184130 20826 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.218.254" port: 43403 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 22:54:12 dist-test-slave-1xrh krb5kdc[20578](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056852, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.252@KRBTEST.COM for kudu/127.28.218.254@KRBTEST.COM
Aug 06 22:54:12 dist-test-slave-1xrh krb5kdc[20578](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056851, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.218.252@KRBTEST.COM
I20260806 22:54:12.193956 20669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:54:12.185096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54402 (local address 127.28.218.254:43403)
0806 22:54:12.185143 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 22:54:12.185145 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:54:12.185242 (+ 97us) server_negotiation.cc:407] Connection header received
0806 22:54:12.185312 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:54:12.185313 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:54:12.185321 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:54:12.185368 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:54:12.185913 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:54:12.186626 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:54:12.187905 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:54:12.188182 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:54:12.188256 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:54:12.190618 (+ 2362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:54:12.190620 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:54:12.190620 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:54:12.190628 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:54:12.192361 (+ 1733us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:54:12.192837 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:54:12.192839 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:54:12.192839 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:54:12.192921 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:54:12.193239 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:54:12.193239 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:54:12.193240 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:54:12.193389 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:54:12.193466 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 22:54:12.193784 (+ 318us) server_negotiation.cc:299] Negotiation successful
0806 22:54:12.193868 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260806 22:54:12.194074 20830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:54:12.184964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:43403 (local address 127.0.0.1:54402)
0806 22:54:12.185121 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0806 22:54:12.185166 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 22:54:12.185248 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:54:12.185428 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:54:12.185431 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:54:12.185728 (+ 297us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:54:12.185823 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:54:12.185828 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:54:12.186703 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:54:12.186704 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:54:12.187825 (+ 1121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:54:12.187827 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:54:12.188291 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:54:12.188292 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:54:12.188364 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:54:12.188927 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:54:12.188937 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:54:12.190511 (+ 1574us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:54:12.192454 (+ 1943us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:54:12.192460 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:54:12.192462 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:54:12.192762 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:54:12.192992 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:54:12.192992 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:54:12.192993 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:54:12.193161 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:54:12.193481 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:54:12.193488 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:54:12.193654 (+ 166us) client_negotiation.cc:769] Sending connection context
0806 22:54:12.193735 (+ 81us) client_negotiation.cc:240] Negotiation successful
0806 22:54:12.193840 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":91,"threads_started":1}
I20260806 22:54:12.194471 20829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:54:12.183781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47230 (local address 127.28.218.252:41547)
0806 22:54:12.183961 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0806 22:54:12.183970 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:54:12.184002 (+ 32us) server_negotiation.cc:407] Connection header received
0806 22:54:12.184387 (+ 385us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:54:12.184393 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:54:12.184443 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:54:12.184550 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:54:12.185568 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:54:12.186772 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:54:12.187811 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:54:12.188229 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:54:12.188325 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:54:12.191102 (+ 2777us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:54:12.191125 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:54:12.191125 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:54:12.191155 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:54:12.192978 (+ 1823us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:54:12.193351 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:54:12.193353 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:54:12.193354 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:54:12.193409 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:54:12.193613 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:54:12.193614 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:54:12.193614 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:54:12.193768 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:54:12.193966 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0806 22:54:12.194266 (+ 300us) server_negotiation.cc:299] Negotiation successful
0806 22:54:12.194359 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":102,"threads_started":1}
I20260806 22:54:12.194711 20826 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.218.253" port: 36185 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 22:54:12 dist-test-slave-1xrh krb5kdc[20578](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056852, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.252@KRBTEST.COM for kudu/127.28.218.253@KRBTEST.COM
Aug 06 22:54:12 dist-test-slave-1xrh krb5kdc[20578](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056851, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.218.254@KRBTEST.COM
I20260806 22:54:12.202427 20830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:54:12.194957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.253:36185 (local address 127.0.0.1:43634)
0806 22:54:12.195200 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0806 22:54:12.195211 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:54:12.195275 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:54:12.195430 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:54:12.195431 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:54:12.195586 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:54:12.195642 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:54:12.195644 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:54:12.196508 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:54:12.196508 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:54:12.197266 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:54:12.197268 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:54:12.197690 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:54:12.197691 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:54:12.197759 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:54:12.198211 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:54:12.198215 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:54:12.199566 (+ 1351us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:54:12.201377 (+ 1811us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:54:12.201378 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:54:12.201379 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:54:12.201601 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:54:12.201819 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:54:12.201820 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:54:12.201820 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:54:12.201891 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:54:12.202156 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:54:12.202157 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:54:12.202203 (+ 46us) client_negotiation.cc:769] Sending connection context
0806 22:54:12.202275 (+ 72us) client_negotiation.cc:240] Negotiation successful
0806 22:54:12.202373 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":209}
I20260806 22:54:12.202518 20750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:54:12.195009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43634 (local address 127.28.218.253:36185)
0806 22:54:12.195060 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0806 22:54:12.195062 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:54:12.195253 (+ 191us) server_negotiation.cc:407] Connection header received
0806 22:54:12.195343 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:54:12.195344 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:54:12.195356 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:54:12.195412 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:54:12.195726 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:54:12.196431 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:54:12.197363 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:54:12.197617 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:54:12.197684 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:54:12.199652 (+ 1968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:54:12.199654 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:54:12.199655 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:54:12.199663 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:54:12.201292 (+ 1629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:54:12.201675 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:54:12.201676 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:54:12.201676 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:54:12.201726 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:54:12.201970 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:54:12.201971 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:54:12.201971 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:54:12.202086 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:54:12.202138 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 22:54:12.202335 (+ 197us) server_negotiation.cc:299] Negotiation successful
0806 22:54:12.202408 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 22:54:12.202816 20826 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.218.252" port: 41547 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:54:12.203469 20669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:54:12.195209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54404 (local address 127.28.218.254:43403)
0806 22:54:12.195265 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0806 22:54:12.195267 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:54:12.195276 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:54:12.195295 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:54:12.195296 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:54:12.195309 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:54:12.195364 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:54:12.195921 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:54:12.196568 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:54:12.197505 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:54:12.197788 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:54:12.197841 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:54:12.200392 (+ 2551us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:54:12.200393 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:54:12.200393 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:54:12.200399 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:54:12.202210 (+ 1811us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:54:12.202610 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:54:12.202611 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:54:12.202612 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:54:12.202670 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:54:12.202867 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:54:12.202868 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:54:12.202868 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:54:12.203015 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:54:12.203248 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0806 22:54:12.203286 (+ 38us) server_negotiation.cc:299] Negotiation successful
0806 22:54:12.203375 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
Aug 06 22:54:12 dist-test-slave-1xrh krb5kdc[20578](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056852, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.252@KRBTEST.COM for kudu/127.28.218.252@KRBTEST.COM
Aug 06 22:54:12 dist-test-slave-1xrh krb5kdc[20578](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056851, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.218.253@KRBTEST.COM
I20260806 22:54:12.209582 20830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:54:12.203051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.252:41547 (local address 127.0.0.1:47234)
0806 22:54:12.203108 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0806 22:54:12.203135 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0806 22:54:12.203178 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:54:12.203292 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:54:12.203293 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:54:12.203467 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:54:12.203498 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:54:12.203500 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:54:12.204177 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:54:12.204178 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:54:12.204738 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:54:12.204740 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:54:12.205120 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:54:12.205121 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:54:12.205167 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:54:12.205537 (+ 370us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:54:12.205542 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:54:12.206955 (+ 1413us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:54:12.208474 (+ 1519us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:54:12.208475 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:54:12.208475 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:54:12.208640 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:54:12.208833 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:54:12.208841 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:54:12.208841 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:54:12.208891 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:54:12.209216 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:54:12.209217 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:54:12.209285 (+ 68us) client_negotiation.cc:769] Sending connection context
0806 22:54:12.209364 (+ 79us) client_negotiation.cc:240] Negotiation successful
0806 22:54:12.209464 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260806 22:54:12.209583 20829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:54:12.203080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47234 (local address 127.28.218.252:41547)
0806 22:54:12.203111 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0806 22:54:12.203114 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:54:12.203164 (+ 50us) server_negotiation.cc:407] Connection header received
0806 22:54:12.203229 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:54:12.203230 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:54:12.203237 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:54:12.203289 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:54:12.203579 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:54:12.204114 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:54:12.204825 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:54:12.205041 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:54:12.205119 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:54:12.207036 (+ 1917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:54:12.207037 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:54:12.207038 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:54:12.207045 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:54:12.208376 (+ 1331us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:54:12.208720 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:54:12.208721 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:54:12.208722 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:54:12.208772 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:54:12.208986 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:54:12.208987 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:54:12.208987 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:54:12.209130 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:54:12.209199 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 22:54:12.209393 (+ 194us) server_negotiation.cc:299] Negotiation successful
0806 22:54:12.209511 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":56,"server-negotiator.queue_time_us":14}
I20260806 22:54:12.211565 20826 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7aa720c12c624118918aff51be3d9a9f: Bootstrap starting.
I20260806 22:54:12.211691 20750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:54:12.204375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43646 (local address 127.28.218.253:36185)
0806 22:54:12.204418 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 22:54:12.204420 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:54:12.204428 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:54:12.204451 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:54:12.204452 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:54:12.204461 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:54:12.204503 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:54:12.204952 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:54:12.205485 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:54:12.206356 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:54:12.206673 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:54:12.206737 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:54:12.208848 (+ 2111us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:54:12.208849 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:54:12.208849 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:54:12.208856 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:54:12.210520 (+ 1664us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:54:12.210825 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:54:12.210826 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:54:12.210826 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:54:12.210882 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:54:12.211066 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:54:12.211067 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:54:12.211067 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:54:12.211208 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:54:12.211384 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0806 22:54:12.211426 (+ 42us) server_negotiation.cc:299] Negotiation successful
0806 22:54:12.211613 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 22:54:12.212611 20826 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7aa720c12c624118918aff51be3d9a9f: Neither blocks nor log segments found. Creating new log.
I20260806 22:54:12.212939 20826 log.cc:824] T 00000000000000000000000000000000 P 7aa720c12c624118918aff51be3d9a9f: Log is configured to *not* fsync() on all Append() calls
I20260806 22:54:12.213064 29547 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:54:12.213688 20826 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7aa720c12c624118918aff51be3d9a9f: No bootstrap required, opened a new log
I20260806 22:54:12.214041 20637 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "67499e7f32e249ca834e2839ad85f46e"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54404
I20260806 22:54:12.215426 20826 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7aa720c12c624118918aff51be3d9a9f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67499e7f32e249ca834e2839ad85f46e" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 43403 } } peers { permanent_uuid: "a5a8a2ee77ca4812b90172945f44652e" member_type: VOTER last_known_addr { host: "127.28.218.253" port: 36185 } } peers { permanent_uuid: "7aa720c12c624118918aff51be3d9a9f" member_type: VOTER last_known_addr { host: "127.28.218.252" port: 41547 } }
I20260806 22:54:12.215569 20826 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7aa720c12c624118918aff51be3d9a9f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aa720c12c624118918aff51be3d9a9f, State: Initialized, Role: FOLLOWER
I20260806 22:54:12.215657 20826 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7aa720c12c624118918aff51be3d9a9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67499e7f32e249ca834e2839ad85f46e" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 43403 } } peers { permanent_uuid: "a5a8a2ee77ca4812b90172945f44652e" member_type: VOTER last_known_addr { host: "127.28.218.253" port: 36185 } } peers { permanent_uuid: "7aa720c12c624118918aff51be3d9a9f" member_type: VOTER last_known_addr { host: "127.28.218.252" port: 41547 } }
I20260806 22:54:12.215909 20832 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7aa720c12c624118918aff51be3d9a9f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67499e7f32e249ca834e2839ad85f46e" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 43403 } } peers { permanent_uuid: "a5a8a2ee77ca4812b90172945f44652e" member_type: VOTER last_known_addr { host: "127.28.218.253" port: 36185 } } peers { permanent_uuid: "7aa720c12c624118918aff51be3d9a9f" member_type: VOTER last_known_addr { host: "127.28.218.252" port: 41547 } } }
I20260806 22:54:12.215991 20826 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7aa720c12c624118918aff51be3d9a9f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:54:12.215996 20832 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7aa720c12c624118918aff51be3d9a9f [sys.catalog]: This master's current role is: FOLLOWER
W20260806 22:54:12.217834 20843 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7aa720c12c624118918aff51be3d9a9f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 22:54:12.217941 20843 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 22:54:12.220075 20666 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.218.252" port: 41547 } attempt: 2
I20260806 22:54:12.217978 20753 master_runner.cc:186] Error getting master registration for 127.28.218.254:43403: OK, instance_id { permanent_uuid: "67499e7f32e249ca834e2839ad85f46e" instance_seqno: 1786056851878808 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 22:54:12.220359 20753 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
Aug 06 22:54:12 dist-test-slave-1xrh krb5kdc[20578](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056851, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.254@KRBTEST.COM for kudu/127.28.218.252@KRBTEST.COM
I20260806 22:54:12.226979 20829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:54:12.220341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47240 (local address 127.28.218.252:41547)
0806 22:54:12.220380 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0806 22:54:12.220382 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:54:12.220391 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:54:12.220455 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:54:12.220456 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:54:12.220464 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:54:12.220518 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:54:12.220788 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:54:12.221438 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:54:12.222065 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:54:12.222248 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:54:12.222315 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:54:12.224265 (+ 1950us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:54:12.224266 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:54:12.224266 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:54:12.224273 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:54:12.225835 (+ 1562us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:54:12.226170 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:54:12.226171 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:54:12.226171 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:54:12.226220 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:54:12.226454 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:54:12.226455 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:54:12.226456 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:54:12.226588 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:54:12.226657 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 22:54:12.226808 (+ 151us) server_negotiation.cc:299] Negotiation successful
0806 22:54:12.226880 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 22:54:12.226984 20668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:54:12.220307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.252:41547 (local address 127.0.0.1:47240)
0806 22:54:12.220356 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0806 22:54:12.220366 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:54:12.220400 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:54:12.220521 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:54:12.220522 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:54:12.220678 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:54:12.220716 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:54:12.220717 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:54:12.221514 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:54:12.221514 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:54:12.221986 (+ 472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:54:12.221987 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:54:12.222314 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:54:12.222314 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:54:12.222369 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:54:12.222731 (+ 362us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:54:12.222735 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:54:12.224171 (+ 1436us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:54:12.225927 (+ 1756us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:54:12.225928 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:54:12.225929 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:54:12.226089 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:54:12.226298 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:54:12.226299 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:54:12.226299 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:54:12.226381 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:54:12.226662 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:54:12.226663 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:54:12.226703 (+ 40us) client_negotiation.cc:769] Sending connection context
0806 22:54:12.226784 (+ 81us) client_negotiation.cc:240] Negotiation successful
0806 22:54:12.226880 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260806 22:54:12.228284 20666 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 67499e7f32e249ca834e2839ad85f46e: Bootstrap starting.
I20260806 22:54:12.228785 20666 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 67499e7f32e249ca834e2839ad85f46e: Neither blocks nor log segments found. Creating new log.
I20260806 22:54:12.229002 20666 log.cc:824] T 00000000000000000000000000000000 P 67499e7f32e249ca834e2839ad85f46e: Log is configured to *not* fsync() on all Append() calls
I20260806 22:54:12.229643 20666 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 67499e7f32e249ca834e2839ad85f46e: No bootstrap required, opened a new log
I20260806 22:54:12.230583 20666 raft_consensus.cc:348] T 00000000000000000000000000000000 P 67499e7f32e249ca834e2839ad85f46e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67499e7f32e249ca834e2839ad85f46e" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 43403 } } peers { permanent_uuid: "a5a8a2ee77ca4812b90172945f44652e" member_type: VOTER last_known_addr { host: "127.28.218.253" port: 36185 } } peers { permanent_uuid: "7aa720c12c624118918aff51be3d9a9f" member_type: VOTER last_known_addr { host: "127.28.218.252" port: 41547 } }
I20260806 22:54:12.230696 20666 raft_consensus.cc:729] T 00000000000000000000000000000000 P 67499e7f32e249ca834e2839ad85f46e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67499e7f32e249ca834e2839ad85f46e, State: Initialized, Role: FOLLOWER
I20260806 22:54:12.230778 20666 consensus_queue.cc:260] T 00000000000000000000000000000000 P 67499e7f32e249ca834e2839ad85f46e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67499e7f32e249ca834e2839ad85f46e" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 43403 } } peers { permanent_uuid: "a5a8a2ee77ca4812b90172945f44652e" member_type: VOTER last_known_addr { host: "127.28.218.253" port: 36185 } } peers { permanent_uuid: "7aa720c12c624118918aff51be3d9a9f" member_type: VOTER last_known_addr { host: "127.28.218.252" port: 41547 } }
I20260806 22:54:12.230974 20846 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67499e7f32e249ca834e2839ad85f46e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67499e7f32e249ca834e2839ad85f46e" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 43403 } } peers { permanent_uuid: "a5a8a2ee77ca4812b90172945f44652e" member_type: VOTER last_known_addr { host: "127.28.218.253" port: 36185 } } peers { permanent_uuid: "7aa720c12c624118918aff51be3d9a9f" member_type: VOTER last_known_addr { host: "127.28.218.252" port: 41547 } } }
I20260806 22:54:12.231022 20666 sys_catalog.cc:564] T 00000000000000000000000000000000 P 67499e7f32e249ca834e2839ad85f46e [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:54:12.231034 20846 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67499e7f32e249ca834e2839ad85f46e [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:54:12.231061 20637 raft_consensus.cc:482] T 00000000000000000000000000000000 P 67499e7f32e249ca834e2839ad85f46e [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260806 22:54:12.231144 20637 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 67499e7f32e249ca834e2839ad85f46e [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:54:12.231832 20637 raft_consensus.cc:504] T 00000000000000000000000000000000 P 67499e7f32e249ca834e2839ad85f46e [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67499e7f32e249ca834e2839ad85f46e" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 43403 } } peers { permanent_uuid: "a5a8a2ee77ca4812b90172945f44652e" member_type: VOTER last_known_addr { host: "127.28.218.253" port: 36185 } } peers { permanent_uuid: "7aa720c12c624118918aff51be3d9a9f" member_type: VOTER last_known_addr { host: "127.28.218.252" port: 41547 } }
W20260806 22:54:12.232613 20858 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 67499e7f32e249ca834e2839ad85f46e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 22:54:12.232674 20858 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 22:54:12.232616 20800 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "67499e7f32e249ca834e2839ad85f46e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "7aa720c12c624118918aff51be3d9a9f"
I20260806 22:54:12.232728 20800 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7aa720c12c624118918aff51be3d9a9f [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:54:12.233559 20800 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7aa720c12c624118918aff51be3d9a9f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 67499e7f32e249ca834e2839ad85f46e in term 1.
I20260806 22:54:12.233806 20613 leader_election.cc:302] T 00000000000000000000000000000000 P 67499e7f32e249ca834e2839ad85f46e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 67499e7f32e249ca834e2839ad85f46e, 7aa720c12c624118918aff51be3d9a9f; no voters:
I20260806 22:54:12.233945 20846 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 67499e7f32e249ca834e2839ad85f46e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:54:12.234087 20846 raft_consensus.cc:686] T 00000000000000000000000000000000 P 67499e7f32e249ca834e2839ad85f46e [term 1 LEADER]: Becoming Leader. State: Replica: 67499e7f32e249ca834e2839ad85f46e, State: Running, Role: LEADER
I20260806 22:54:12.234165 20846 consensus_queue.cc:237] T 00000000000000000000000000000000 P 67499e7f32e249ca834e2839ad85f46e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67499e7f32e249ca834e2839ad85f46e" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 43403 } } peers { permanent_uuid: "a5a8a2ee77ca4812b90172945f44652e" member_type: VOTER last_known_addr { host: "127.28.218.253" port: 36185 } } peers { permanent_uuid: "7aa720c12c624118918aff51be3d9a9f" member_type: VOTER last_known_addr { host: "127.28.218.252" port: 41547 } }
I20260806 22:54:12.234580 20860 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67499e7f32e249ca834e2839ad85f46e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 67499e7f32e249ca834e2839ad85f46e. Latest consensus state: current_term: 1 leader_uuid: "67499e7f32e249ca834e2839ad85f46e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67499e7f32e249ca834e2839ad85f46e" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 43403 } } peers { permanent_uuid: "a5a8a2ee77ca4812b90172945f44652e" member_type: VOTER last_known_addr { host: "127.28.218.253" port: 36185 } } peers { permanent_uuid: "7aa720c12c624118918aff51be3d9a9f" member_type: VOTER last_known_addr { host: "127.28.218.252" port: 41547 } } }
I20260806 22:54:12.234664 20637 leader_election.cc:288] T 00000000000000000000000000000000 P 67499e7f32e249ca834e2839ad85f46e [CANDIDATE]: Term 1 election: Requested vote from peers a5a8a2ee77ca4812b90172945f44652e (127.28.218.253:36185), 7aa720c12c624118918aff51be3d9a9f (127.28.218.252:41547)
I20260806 22:54:12.233525 20591 master_runner.cc:186] Error getting master registration for 127.28.218.253:36185: OK, instance_id { permanent_uuid: "a5a8a2ee77ca4812b90172945f44652e" instance_seqno: 1786056852024835 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 22:54:12.235692 20591 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 22:54:12.234756 20860 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67499e7f32e249ca834e2839ad85f46e [sys.catalog]: This master's current role is: LEADER
I20260806 22:54:12.236227 20867 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:54:12.236601 20867 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:54:12.239252 20800 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7aa720c12c624118918aff51be3d9a9f [term 1 FOLLOWER]: Refusing update from remote peer 67499e7f32e249ca834e2839ad85f46e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:54:12.239588 20846 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 67499e7f32e249ca834e2839ad85f46e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aa720c12c624118918aff51be3d9a9f" member_type: VOTER last_known_addr { host: "127.28.218.252" port: 41547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:54:12.240819 20832 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7aa720c12c624118918aff51be3d9a9f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 67499e7f32e249ca834e2839ad85f46e. Latest consensus state: current_term: 1 leader_uuid: "67499e7f32e249ca834e2839ad85f46e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67499e7f32e249ca834e2839ad85f46e" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 43403 } } peers { permanent_uuid: "a5a8a2ee77ca4812b90172945f44652e" member_type: VOTER last_known_addr { host: "127.28.218.253" port: 36185 } } peers { permanent_uuid: "7aa720c12c624118918aff51be3d9a9f" member_type: VOTER last_known_addr { host: "127.28.218.252" port: 41547 } } }
I20260806 22:54:12.240911 20832 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7aa720c12c624118918aff51be3d9a9f [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:54:12.242568 20832 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7aa720c12c624118918aff51be3d9a9f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "67499e7f32e249ca834e2839ad85f46e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67499e7f32e249ca834e2839ad85f46e" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 43403 } } peers { permanent_uuid: "a5a8a2ee77ca4812b90172945f44652e" member_type: VOTER last_known_addr { host: "127.28.218.253" port: 36185 } } peers { permanent_uuid: "7aa720c12c624118918aff51be3d9a9f" member_type: VOTER last_known_addr { host: "127.28.218.252" port: 41547 } } }
I20260806 22:54:12.242642 20832 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7aa720c12c624118918aff51be3d9a9f [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:54:12.242967 20860 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67499e7f32e249ca834e2839ad85f46e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "67499e7f32e249ca834e2839ad85f46e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67499e7f32e249ca834e2839ad85f46e" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 43403 } } peers { permanent_uuid: "a5a8a2ee77ca4812b90172945f44652e" member_type: VOTER last_known_addr { host: "127.28.218.253" port: 36185 } } peers { permanent_uuid: "7aa720c12c624118918aff51be3d9a9f" member_type: VOTER last_known_addr { host: "127.28.218.252" port: 41547 } } }
I20260806 22:54:12.243059 20860 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67499e7f32e249ca834e2839ad85f46e [sys.catalog]: This master's current role is: LEADER
I20260806 22:54:12.243170 20869 mvcc.cc:205] Tried to move back new op lower bound from 7315688866766553088 to 7315688866751725568. Current Snapshot: MvccSnapshot[applied={T|T < 7315688866766553088}]
I20260806 22:54:12.243331 20870 mvcc.cc:205] Tried to move back new op lower bound from 7315688866766553088 to 7315688866751725568. Current Snapshot: MvccSnapshot[applied={T|T < 7315688866766553088}]
I20260806 22:54:12.243844 20867 catalog_manager.cc:1347] Generated new cluster ID: 46ad78858082466c90b288b9bfc3b97c
I20260806 22:54:12.243891 20867 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:54:12.247375 20829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:54:12.236258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47246 (local address 127.28.218.252:41547)
0806 22:54:12.236287 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0806 22:54:12.236288 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:54:12.237548 (+ 1260us) server_negotiation.cc:407] Connection header received
0806 22:54:12.237645 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:54:12.237646 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:54:12.237655 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:54:12.237684 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:54:12.238451 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:54:12.239123 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:54:12.240429 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:54:12.240701 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:54:12.240746 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:54:12.244766 (+ 4020us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:54:12.244767 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:54:12.244767 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:54:12.244773 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:54:12.246223 (+ 1450us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:54:12.246556 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:54:12.246557 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:54:12.246557 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:54:12.246602 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:54:12.246747 (+ 145us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:54:12.246747 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:54:12.246748 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:54:12.246841 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:54:12.246906 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 22:54:12.247220 (+ 314us) server_negotiation.cc:299] Negotiation successful
0806 22:54:12.247288 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260806 22:54:12.247478 20669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:54:12.235888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54416 (local address 127.28.218.254:43403)
0806 22:54:12.235938 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 22:54:12.235940 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:54:12.236070 (+ 130us) server_negotiation.cc:407] Connection header received
0806 22:54:12.236162 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:54:12.236163 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:54:12.236172 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:54:12.236202 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:54:12.236939 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:54:12.237653 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:54:12.238691 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:54:12.239042 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:54:12.239100 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:54:12.240663 (+ 1563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:54:12.240665 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:54:12.240665 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:54:12.240672 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:54:12.242309 (+ 1637us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:54:12.242686 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:54:12.242687 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:54:12.242687 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:54:12.242758 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:54:12.243002 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:54:12.243003 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:54:12.243003 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:54:12.243172 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:54:12.243268 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0806 22:54:12.243700 (+ 432us) server_negotiation.cc:299] Negotiation successful
0806 22:54:12.243867 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 22:54:12.251750 20750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:54:12.239306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43660 (local address 127.28.218.253:36185)
0806 22:54:12.239359 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0806 22:54:12.239360 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:54:12.241803 (+ 2443us) server_negotiation.cc:407] Connection header received
0806 22:54:12.241942 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:54:12.241944 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:54:12.241965 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:54:12.241994 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:54:12.244596 (+ 2602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:54:12.245307 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:54:12.246691 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:54:12.247006 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:54:12.247045 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:54:12.248782 (+ 1737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:54:12.248784 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:54:12.248784 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:54:12.248790 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:54:12.250398 (+ 1608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:54:12.250823 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:54:12.250824 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:54:12.250825 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:54:12.250889 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:54:12.251099 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:54:12.251100 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:54:12.251100 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:54:12.251242 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:54:12.251301 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 22:54:12.251588 (+ 287us) server_negotiation.cc:299] Negotiation successful
0806 22:54:12.251667 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 22:54:12.258898 20745 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.218.252" port: 41547 } attempt: 3
I20260806 22:54:12.261502 20867 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:54:12.262204 20867 catalog_manager.cc:1504] Loading token signing keys...
Aug 06 22:54:12 dist-test-slave-1xrh krb5kdc[20578](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056852, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.253@KRBTEST.COM for kudu/127.28.218.252@KRBTEST.COM
I20260806 22:54:12.266471 20747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:54:12.259109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.252:41547 (local address 127.0.0.1:47254)
0806 22:54:12.259200 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0806 22:54:12.259209 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:54:12.259245 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:54:12.259463 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:54:12.259464 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:54:12.259613 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:54:12.259660 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:54:12.259661 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:54:12.260508 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:54:12.260509 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:54:12.261277 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:54:12.261279 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:54:12.261718 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:54:12.261719 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:54:12.261788 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:54:12.262254 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:54:12.262259 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:54:12.263737 (+ 1478us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:54:12.265478 (+ 1741us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:54:12.265479 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:54:12.265480 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:54:12.265674 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:54:12.265890 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:54:12.265890 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:54:12.265891 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:54:12.265952 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:54:12.266209 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:54:12.266210 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:54:12.266259 (+ 49us) client_negotiation.cc:769] Sending connection context
0806 22:54:12.266310 (+ 51us) client_negotiation.cc:240] Negotiation successful
0806 22:54:12.266387 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":76}
I20260806 22:54:12.266956 20829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:54:12.259308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47254 (local address 127.28.218.252:41547)
0806 22:54:12.259352 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 22:54:12.259354 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:54:12.259360 (+ 6us) server_negotiation.cc:407] Connection header received
0806 22:54:12.259377 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:54:12.259378 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:54:12.259385 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:54:12.259429 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:54:12.259733 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:54:12.260423 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:54:12.261354 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:54:12.261636 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:54:12.261718 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:54:12.263843 (+ 2125us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:54:12.263844 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:54:12.263844 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:54:12.263851 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:54:12.265387 (+ 1536us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:54:12.265748 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:54:12.265749 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:54:12.265749 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:54:12.265807 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:54:12.266016 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:54:12.266016 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:54:12.266017 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:54:12.266128 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:54:12.266174 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0806 22:54:12.266810 (+ 636us) server_negotiation.cc:299] Negotiation successful
0806 22:54:12.266881 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 22:54:12.268251 20745 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a5a8a2ee77ca4812b90172945f44652e: Bootstrap starting.
I20260806 22:54:12.268865 20745 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a5a8a2ee77ca4812b90172945f44652e: Neither blocks nor log segments found. Creating new log.
I20260806 22:54:12.269102 20745 log.cc:824] T 00000000000000000000000000000000 P a5a8a2ee77ca4812b90172945f44652e: Log is configured to *not* fsync() on all Append() calls
I20260806 22:54:12.277474 20867 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 67499e7f32e249ca834e2839ad85f46e: Generated new TSK 0
I20260806 22:54:12.277635 20745 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a5a8a2ee77ca4812b90172945f44652e: No bootstrap required, opened a new log
I20260806 22:54:12.277812 20867 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:54:12.279701 20745 raft_consensus.cc:348] T 00000000000000000000000000000000 P a5a8a2ee77ca4812b90172945f44652e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67499e7f32e249ca834e2839ad85f46e" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 43403 } } peers { permanent_uuid: "a5a8a2ee77ca4812b90172945f44652e" member_type: VOTER last_known_addr { host: "127.28.218.253" port: 36185 } } peers { permanent_uuid: "7aa720c12c624118918aff51be3d9a9f" member_type: VOTER last_known_addr { host: "127.28.218.252" port: 41547 } }
I20260806 22:54:12.279825 20745 raft_consensus.cc:729] T 00000000000000000000000000000000 P a5a8a2ee77ca4812b90172945f44652e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5a8a2ee77ca4812b90172945f44652e, State: Initialized, Role: FOLLOWER
I20260806 22:54:12.279904 20745 consensus_queue.cc:260] T 00000000000000000000000000000000 P a5a8a2ee77ca4812b90172945f44652e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67499e7f32e249ca834e2839ad85f46e" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 43403 } } peers { permanent_uuid: "a5a8a2ee77ca4812b90172945f44652e" member_type: VOTER last_known_addr { host: "127.28.218.253" port: 36185 } } peers { permanent_uuid: "7aa720c12c624118918aff51be3d9a9f" member_type: VOTER last_known_addr { host: "127.28.218.252" port: 41547 } }
I20260806 22:54:12.280215 20745 sys_catalog.cc:564] T 00000000000000000000000000000000 P a5a8a2ee77ca4812b90172945f44652e [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:54:12.280252 20717 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "67499e7f32e249ca834e2839ad85f46e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "a5a8a2ee77ca4812b90172945f44652e"
I20260806 22:54:12.280565 20717 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a5a8a2ee77ca4812b90172945f44652e [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:54:12.281520 20717 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a5a8a2ee77ca4812b90172945f44652e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 67499e7f32e249ca834e2839ad85f46e in term 1.
I20260806 22:54:12.280179 20877 sys_catalog.cc:455] T 00000000000000000000000000000000 P a5a8a2ee77ca4812b90172945f44652e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67499e7f32e249ca834e2839ad85f46e" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 43403 } } peers { permanent_uuid: "a5a8a2ee77ca4812b90172945f44652e" member_type: VOTER last_known_addr { host: "127.28.218.253" port: 36185 } } peers { permanent_uuid: "7aa720c12c624118918aff51be3d9a9f" member_type: VOTER last_known_addr { host: "127.28.218.252" port: 41547 } } }
I20260806 22:54:12.281899 20877 sys_catalog.cc:458] T 00000000000000000000000000000000 P a5a8a2ee77ca4812b90172945f44652e [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:54:12.281999 20716 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a5a8a2ee77ca4812b90172945f44652e [term 1 FOLLOWER]: Refusing update from remote peer 67499e7f32e249ca834e2839ad85f46e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:54:12.282235 20860 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 67499e7f32e249ca834e2839ad85f46e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a5a8a2ee77ca4812b90172945f44652e" member_type: VOTER last_known_addr { host: "127.28.218.253" port: 36185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:54:12.284443 20877 sys_catalog.cc:455] T 00000000000000000000000000000000 P a5a8a2ee77ca4812b90172945f44652e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 67499e7f32e249ca834e2839ad85f46e. Latest consensus state: current_term: 1 leader_uuid: "67499e7f32e249ca834e2839ad85f46e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67499e7f32e249ca834e2839ad85f46e" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 43403 } } peers { permanent_uuid: "a5a8a2ee77ca4812b90172945f44652e" member_type: VOTER last_known_addr { host: "127.28.218.253" port: 36185 } } peers { permanent_uuid: "7aa720c12c624118918aff51be3d9a9f" member_type: VOTER last_known_addr { host: "127.28.218.252" port: 41547 } } }
I20260806 22:54:12.284763 20877 sys_catalog.cc:458] T 00000000000000000000000000000000 P a5a8a2ee77ca4812b90172945f44652e [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:54:12.284578 20891 sys_catalog.cc:455] T 00000000000000000000000000000000 P a5a8a2ee77ca4812b90172945f44652e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "67499e7f32e249ca834e2839ad85f46e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67499e7f32e249ca834e2839ad85f46e" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 43403 } } peers { permanent_uuid: "a5a8a2ee77ca4812b90172945f44652e" member_type: VOTER last_known_addr { host: "127.28.218.253" port: 36185 } } peers { permanent_uuid: "7aa720c12c624118918aff51be3d9a9f" member_type: VOTER last_known_addr { host: "127.28.218.252" port: 41547 } } }
I20260806 22:54:12.284929 20891 sys_catalog.cc:458] T 00000000000000000000000000000000 P a5a8a2ee77ca4812b90172945f44652e [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:54:12.295670 20888 catalog_manager.cc:1259] Loaded cluster ID: 46ad78858082466c90b288b9bfc3b97c
I20260806 22:54:12.295720 20888 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a5a8a2ee77ca4812b90172945f44652e: loading cluster ID for follower catalog manager: success
I20260806 22:54:12.296766 20888 catalog_manager.cc:1574] T 00000000000000000000000000000000 P a5a8a2ee77ca4812b90172945f44652e: acquiring CA information for follower catalog manager: success
I20260806 22:54:12.297050 20888 catalog_manager.cc:1602] T 00000000000000000000000000000000 P a5a8a2ee77ca4812b90172945f44652e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260806 22:54:12.301630 20829 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:54:12.300604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47266 (local address 127.28.218.252:41547)
0806 22:54:12.300640 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0806 22:54:12.300642 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:54:12.301076 (+ 434us) server_negotiation.cc:407] Connection header received
0806 22:54:12.301100 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:54:12.301101 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:54:12.301110 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:54:12.301156 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:54:12.301571 (+ 415us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47266: BlockingRecv error: recv got EOF from 127.0.0.1:47266 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260806 22:54:12.301723 20669 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:54:12.300465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54420 (local address 127.28.218.254:43403)
0806 22:54:12.300510 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 22:54:12.300512 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:54:12.301177 (+ 665us) server_negotiation.cc:407] Connection header received
0806 22:54:12.301269 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:54:12.301270 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:54:12.301281 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:54:12.301323 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:54:12.301655 (+ 332us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54420: BlockingRecv error: recv got EOF from 127.0.0.1:54420 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260806 22:54:12.302111 20750 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:54:12.300202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43668 (local address 127.28.218.253:36185)
0806 22:54:12.300239 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0806 22:54:12.300241 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:54:12.300895 (+ 654us) server_negotiation.cc:407] Connection header received
0806 22:54:12.300971 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:54:12.300972 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:54:12.300982 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:54:12.301046 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:54:12.302067 (+ 1021us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43668: BlockingRecv error: recv got EOF from 127.0.0.1:43668 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 22:54:12.306766 20750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:54:12.303490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43676 (local address 127.28.218.253:36185)
0806 22:54:12.303522 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0806 22:54:12.303524 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:54:12.303746 (+ 222us) server_negotiation.cc:407] Connection header received
0806 22:54:12.303870 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:54:12.303871 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:54:12.303898 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:54:12.303939 (+ 41us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:54:12.304128 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:54:12.304873 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:54:12.305941 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:54:12.306226 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:54:12.306261 (+ 35us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:54:12.306464 (+ 203us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:54:12.306559 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:54:12.306600 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0806 22:54:12.306700 (+ 100us) server_negotiation.cc:299] Negotiation successful
0806 22:54:12.306714 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
W20260806 22:54:12.309448 20829 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:54:12.308783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47282 (local address 127.28.218.252:41547)
0806 22:54:12.308814 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0806 22:54:12.308816 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:54:12.308993 (+ 177us) server_negotiation.cc:407] Connection header received
0806 22:54:12.309080 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:54:12.309081 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:54:12.309095 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:54:12.309123 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:54:12.309404 (+ 281us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47282: BlockingRecv error: recv got EOF from 127.0.0.1:47282 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
I20260806 22:54:12.314546 20750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:54:12.311485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43684 (local address 127.28.218.253:36185)
0806 22:54:12.311512 (+ 27us) server_negotiation.cc:206] Beginning negotiation
0806 22:54:12.311513 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:54:12.311718 (+ 205us) server_negotiation.cc:407] Connection header received
0806 22:54:12.311799 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:54:12.311799 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:54:12.311807 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:54:12.311831 (+ 24us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:54:12.311985 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:54:12.312694 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:54:12.313781 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:54:12.314046 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:54:12.314090 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:54:12.314309 (+ 219us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:54:12.314347 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:54:12.314374 (+ 27us) server_negotiation.cc:1035] Waiting for connection context
0806 22:54:12.314475 (+ 101us) server_negotiation.cc:299] Negotiation successful
0806 22:54:12.314491 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
W20260806 22:54:12.317711 20829 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:54:12.316946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47288 (local address 127.28.218.252:41547)
0806 22:54:12.316979 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0806 22:54:12.316981 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:54:12.317178 (+ 197us) server_negotiation.cc:407] Connection header received
0806 22:54:12.317274 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:54:12.317275 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:54:12.317285 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:54:12.317316 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:54:12.317667 (+ 351us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47288: BlockingRecv error: recv got EOF from 127.0.0.1:47288 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 22:54:12.323762 20750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:54:12.320769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43694 (local address 127.28.218.253:36185)
0806 22:54:12.320801 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0806 22:54:12.320802 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:54:12.321005 (+ 203us) server_negotiation.cc:407] Connection header received
0806 22:54:12.321097 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:54:12.321098 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:54:12.321105 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:54:12.321132 (+ 27us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:54:12.321285 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:54:12.321954 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:54:12.323000 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:54:12.323294 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:54:12.323348 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:54:12.323535 (+ 187us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:54:12.323564 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:54:12.323591 (+ 27us) server_negotiation.cc:1035] Waiting for connection context
0806 22:54:12.323691 (+ 100us) server_negotiation.cc:299] Negotiation successful
0806 22:54:12.323706 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
W20260806 22:54:12.326655 20829 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:54:12.325875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47304 (local address 127.28.218.252:41547)
0806 22:54:12.325907 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0806 22:54:12.325908 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:54:12.326125 (+ 217us) server_negotiation.cc:407] Connection header received
0806 22:54:12.326217 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:54:12.326218 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:54:12.326228 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:54:12.326255 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:54:12.326564 (+ 309us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47304: BlockingRecv error: recv got EOF from 127.0.0.1:47304 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
I20260806 22:54:12.335067 20750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:54:12.331916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43702 (local address 127.28.218.253:36185)
0806 22:54:12.331949 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0806 22:54:12.331951 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:54:12.332213 (+ 262us) server_negotiation.cc:407] Connection header received
0806 22:54:12.332320 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:54:12.332321 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:54:12.332329 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:54:12.332371 (+ 42us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:54:12.332551 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:54:12.333202 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:54:12.334234 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:54:12.334515 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:54:12.334561 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:54:12.334803 (+ 242us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:54:12.334834 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:54:12.334864 (+ 30us) server_negotiation.cc:1035] Waiting for connection context
0806 22:54:12.334972 (+ 108us) server_negotiation.cc:299] Negotiation successful
0806 22:54:12.334989 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
W20260806 22:54:12.338084 20829 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:54:12.337329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47318 (local address 127.28.218.252:41547)
0806 22:54:12.337380 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0806 22:54:12.337382 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:54:12.337592 (+ 210us) server_negotiation.cc:407] Connection header received
0806 22:54:12.337686 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:54:12.337687 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:54:12.337697 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:54:12.337726 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:54:12.338040 (+ 314us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47318: BlockingRecv error: recv got EOF from 127.0.0.1:47318 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
I20260806 22:54:12.350247 20750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:54:12.347173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43704 (local address 127.28.218.253:36185)
0806 22:54:12.347208 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0806 22:54:12.347209 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:54:12.347444 (+ 235us) server_negotiation.cc:407] Connection header received
0806 22:54:12.347538 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:54:12.347539 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:54:12.347549 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:54:12.347577 (+ 28us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:54:12.347732 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:54:12.348423 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:54:12.349495 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:54:12.349783 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:54:12.349824 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:54:12.350016 (+ 192us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:54:12.350043 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:54:12.350070 (+ 27us) server_negotiation.cc:1035] Waiting for connection context
0806 22:54:12.350171 (+ 101us) server_negotiation.cc:299] Negotiation successful
0806 22:54:12.350186 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
W20260806 22:54:12.353017 20829 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:54:12.352216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47334 (local address 127.28.218.252:41547)
0806 22:54:12.352248 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0806 22:54:12.352250 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:54:12.352507 (+ 257us) server_negotiation.cc:407] Connection header received
0806 22:54:12.352599 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:54:12.352600 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:54:12.352610 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:54:12.352637 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:54:12.352975 (+ 338us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47334: BlockingRecv error: recv got EOF from 127.0.0.1:47334 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
I20260806 22:54:12.373495 20750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:54:12.370249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43714 (local address 127.28.218.253:36185)
0806 22:54:12.370292 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 22:54:12.370294 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:54:12.370583 (+ 289us) server_negotiation.cc:407] Connection header received
0806 22:54:12.370688 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:54:12.370690 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:54:12.370700 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:54:12.370729 (+ 29us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:54:12.370919 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:54:12.371643 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:54:12.372702 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:54:12.372985 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:54:12.373021 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:54:12.373219 (+ 198us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:54:12.373252 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:54:12.373280 (+ 28us) server_negotiation.cc:1035] Waiting for connection context
0806 22:54:12.373398 (+ 118us) server_negotiation.cc:299] Negotiation successful
0806 22:54:12.373420 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
W20260806 22:54:12.376695 20829 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:54:12.375857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47346 (local address 127.28.218.252:41547)
0806 22:54:12.375900 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 22:54:12.375902 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:54:12.376116 (+ 214us) server_negotiation.cc:407] Connection header received
0806 22:54:12.376211 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:54:12.376212 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:54:12.376223 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:54:12.376253 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:54:12.376639 (+ 386us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47346: BlockingRecv error: recv got EOF from 127.0.0.1:47346 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 22:54:12.412669 20750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:54:12.410225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43726 (local address 127.28.218.253:36185)
0806 22:54:12.410273 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 22:54:12.410275 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:54:12.410415 (+ 140us) server_negotiation.cc:407] Connection header received
0806 22:54:12.410495 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:54:12.410496 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:54:12.410507 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:54:12.410575 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:54:12.410715 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:54:12.411240 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:54:12.411987 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:54:12.412177 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:54:12.412253 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:54:12.412394 (+ 141us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:54:12.412432 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:54:12.412492 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 22:54:12.412553 (+ 61us) server_negotiation.cc:299] Negotiation successful
0806 22:54:12.412572 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
W20260806 22:54:12.416036 20829 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:54:12.415381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47350 (local address 127.28.218.252:41547)
0806 22:54:12.415448 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0806 22:54:12.415450 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:54:12.415484 (+ 34us) server_negotiation.cc:407] Connection header received
0806 22:54:12.415596 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:54:12.415597 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:54:12.415608 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:54:12.415654 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:54:12.415977 (+ 323us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47350: BlockingRecv error: recv got EOF from 127.0.0.1:47350 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 22:54:12.483915 20750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:54:12.481375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43742 (local address 127.28.218.253:36185)
0806 22:54:12.481461 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0806 22:54:12.481463 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:54:12.481580 (+ 117us) server_negotiation.cc:407] Connection header received
0806 22:54:12.481667 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:54:12.481668 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:54:12.481678 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:54:12.481724 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:54:12.481890 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:54:12.482364 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:54:12.483177 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:54:12.483381 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:54:12.483472 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:54:12.483635 (+ 163us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:54:12.483674 (+ 39us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:54:12.483732 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 22:54:12.483798 (+ 66us) server_negotiation.cc:299] Negotiation successful
0806 22:54:12.483827 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
W20260806 22:54:12.486038 20829 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:54:12.485549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47356 (local address 127.28.218.252:41547)
0806 22:54:12.485605 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0806 22:54:12.485607 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:54:12.485635 (+ 28us) server_negotiation.cc:407] Connection header received
0806 22:54:12.485698 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:54:12.485699 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:54:12.485708 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:54:12.485748 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:54:12.485995 (+ 247us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47356: BlockingRecv error: recv got EOF from 127.0.0.1:47356 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 22:54:12.617714 20750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:54:12.615238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43756 (local address 127.28.218.253:36185)
0806 22:54:12.615353 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0806 22:54:12.615355 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:54:12.615451 (+ 96us) server_negotiation.cc:407] Connection header received
0806 22:54:12.615536 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:54:12.615537 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:54:12.615548 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:54:12.615593 (+ 45us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:54:12.615712 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:54:12.616262 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:54:12.617057 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:54:12.617267 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:54:12.617339 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:54:12.617499 (+ 160us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:54:12.617533 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:54:12.617588 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 22:54:12.617637 (+ 49us) server_negotiation.cc:299] Negotiation successful
0806 22:54:12.617655 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
W20260806 22:54:12.620105 20829 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:54:12.619481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47364 (local address 127.28.218.252:41547)
0806 22:54:12.619546 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0806 22:54:12.619548 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:54:12.619618 (+ 70us) server_negotiation.cc:407] Connection header received
0806 22:54:12.619657 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:54:12.619658 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:54:12.619667 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:54:12.619716 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:54:12.620029 (+ 313us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47364: BlockingRecv error: recv got EOF from 127.0.0.1:47364 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 22:54:12.761977 20860 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67499e7f32e249ca834e2839ad85f46e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "67499e7f32e249ca834e2839ad85f46e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67499e7f32e249ca834e2839ad85f46e" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 43403 } } peers { permanent_uuid: "a5a8a2ee77ca4812b90172945f44652e" member_type: VOTER last_known_addr { host: "127.28.218.253" port: 36185 } } peers { permanent_uuid: "7aa720c12c624118918aff51be3d9a9f" member_type: VOTER last_known_addr { host: "127.28.218.252" port: 41547 } } }
I20260806 22:54:12.762161 20860 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67499e7f32e249ca834e2839ad85f46e [sys.catalog]: This master's current role is: LEADER
I20260806 22:54:12.879819 20750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:54:12.877377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43766 (local address 127.28.218.253:36185)
0806 22:54:12.877468 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0806 22:54:12.877470 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:54:12.877481 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:54:12.877552 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:54:12.877553 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:54:12.877564 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:54:12.877647 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:54:12.877804 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:54:12.878288 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:54:12.879056 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:54:12.879286 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:54:12.879377 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:54:12.879557 (+ 180us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:54:12.879599 (+ 42us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:54:12.879651 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 22:54:12.879710 (+ 59us) server_negotiation.cc:299] Negotiation successful
0806 22:54:12.879729 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
W20260806 22:54:12.882467 20829 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:54:12.881720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47370 (local address 127.28.218.252:41547)
0806 22:54:12.881772 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0806 22:54:12.881774 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:54:12.881899 (+ 125us) server_negotiation.cc:407] Connection header received
0806 22:54:12.881953 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:54:12.881954 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:54:12.881965 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:54:12.882021 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:54:12.882405 (+ 384us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47370: BlockingRecv error: recv got EOF from 127.0.0.1:47370 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
I20260806 22:54:13.218393 20843 catalog_manager.cc:1259] Loaded cluster ID: 46ad78858082466c90b288b9bfc3b97c
I20260806 22:54:13.218468 20843 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 7aa720c12c624118918aff51be3d9a9f: loading cluster ID for follower catalog manager: success
I20260806 22:54:13.219334 20843 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 7aa720c12c624118918aff51be3d9a9f: acquiring CA information for follower catalog manager: success
I20260806 22:54:13.219604 20843 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 7aa720c12c624118918aff51be3d9a9f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 22:54:13.398187 21016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:54:13.395636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43776 (local address 127.28.218.253:36185)
0806 22:54:13.395770 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0806 22:54:13.395773 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:54:13.395875 (+ 102us) server_negotiation.cc:407] Connection header received
0806 22:54:13.395960 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:54:13.395961 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:54:13.395973 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:54:13.396047 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:54:13.396200 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:54:13.396700 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:54:13.397484 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:54:13.397678 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:54:13.397750 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:54:13.397909 (+ 159us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:54:13.397938 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:54:13.397995 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 22:54:13.398048 (+ 53us) server_negotiation.cc:299] Negotiation successful
0806 22:54:13.398071 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":77,"threads_started":1}
I20260806 22:54:13.402045 21022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:54:13.399880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47380 (local address 127.28.218.252:41547)
0806 22:54:13.400028 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0806 22:54:13.400030 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:54:13.400058 (+ 28us) server_negotiation.cc:407] Connection header received
0806 22:54:13.400078 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:54:13.400079 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:54:13.400125 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:54:13.400183 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:54:13.400281 (+ 98us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:54:13.400750 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:54:13.401372 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:54:13.401553 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:54:13.401600 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:54:13.401719 (+ 119us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:54:13.401819 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:54:13.401874 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 22:54:13.401915 (+ 41us) server_negotiation.cc:299] Negotiation successful
0806 22:54:13.401931 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":89,"threads_started":1}
I20260806 22:54:13.406658 21022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:54:13.403672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47394 (local address 127.28.218.252:41547)
0806 22:54:13.403746 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0806 22:54:13.403748 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:54:13.403911 (+ 163us) server_negotiation.cc:407] Connection header received
0806 22:54:13.403991 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:54:13.403992 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:54:13.404002 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:54:13.404056 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:54:13.404209 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:54:13.404917 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:54:13.405868 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:54:13.406152 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:54:13.406212 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:54:13.406382 (+ 170us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:54:13.406415 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:54:13.406464 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 22:54:13.406556 (+ 92us) server_negotiation.cc:299] Negotiation successful
0806 22:54:13.406584 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 22:54:13.406667 21016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:54:13.403628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43792 (local address 127.28.218.253:36185)
0806 22:54:13.403777 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0806 22:54:13.403779 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:54:13.404013 (+ 234us) server_negotiation.cc:407] Connection header received
0806 22:54:13.404097 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:54:13.404098 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:54:13.404109 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:54:13.404138 (+ 29us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:54:13.404302 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:54:13.404752 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:54:13.405853 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:54:13.406064 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:54:13.406095 (+ 31us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:54:13.406290 (+ 195us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:54:13.406327 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:54:13.406379 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 22:54:13.406557 (+ 178us) server_negotiation.cc:299] Negotiation successful
0806 22:54:13.406579 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131}
I20260806 22:54:13.407081 21028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:54:13.403692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54428 (local address 127.28.218.254:43403)
0806 22:54:13.403852 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0806 22:54:13.403854 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:54:13.403867 (+ 13us) server_negotiation.cc:407] Connection header received
0806 22:54:13.403996 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:54:13.403998 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:54:13.404052 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:54:13.404132 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:54:13.404262 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:54:13.405038 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:54:13.405794 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:54:13.406098 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:54:13.406160 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:54:13.406389 (+ 229us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:54:13.406501 (+ 112us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:54:13.406562 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 22:54:13.406946 (+ 384us) server_negotiation.cc:299] Negotiation successful
0806 22:54:13.406998 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":53,"threads_started":1}
I20260806 22:54:13.408660 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 20591
I20260806 22:54:13.415659 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 20673
I20260806 22:54:13.421991 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 20753
2026-08-06T22:54:13Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (2833 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056586788339-29547-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:54:13 dist-test-slave-1xrh krb5kdc[21033](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:54:13 dist-test-slave-1xrh krb5kdc[21033](info): set up 2 sockets
Aug 06 22:54:13 dist-test-slave-1xrh krb5kdc[21033](info): commencing operation
krb5kdc: starting...
W20260806 22:54:14.467756 29547 mini_kdc.cc:121] Time spent starting KDC: real 1.025s user 0.006s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056586788339-29547-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056586788339-29547-0/krb5kdc/test-user.keytab.
Aug 06 22:54:14 dist-test-slave-1xrh krb5kdc[21033](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056854, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:54:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:54:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.218.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.218.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.218.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab.
Entry for principal kudu/127.28.218.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab.
I20260806 22:54:14.582160 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056586788339-29547-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056586788339-29547-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056586788339-29547-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.218.254:34875
--webserver_interface=127.28.218.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab
--principal=kudu/127.28.218.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.218.212:41199
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.218.254:34875,127.28.218.253:45491,127.28.218.252:40877
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:54:14.660549 21046 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 22:54:14.660768 21046 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:54:14.660805 21046 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:54:14.660848 21046 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:54:14.661927 21046 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:54:14.661993 21046 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:54:14.662037 21046 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:54:14.662058 21046 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:54:14.662087 21046 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:54:14.663591 21046 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:41199
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056586788339-29547-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056586788339-29547-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.218.254:34875,127.28.218.253:45491,127.28.218.252:40877
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.254.keytab
--principal=kudu/127.28.218.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.218.254:34875
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.218.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21046
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056586788339-29547-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:54:14.663851 21046 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:54:14.664063 21046 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:54:14.668896 21051 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:54:14.668926 21052 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:54:14.669052 21046 server_base.cc:1034] running on GCE node
W20260806 22:54:14.668903 21054 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:54:14.669317 21046 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:54:14.669555 21046 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:54:14.670708 21046 hybrid_clock.cc:648] HybridClock initialized: now 1786056854670692 us; error 35 us; skew 500 ppm
Aug 06 22:54:14 dist-test-slave-1xrh krb5kdc[21033](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056854, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:54:14.678261 21046 init.cc:377] Logged in from keytab as kudu/127.28.218.254@KRBTEST.COM (short username kudu)
I20260806 22:54:14.678714 21046 webserver.cc:460] Webserver started at http://127.28.218.254:46623/ using document root <none> and password file <none>
I20260806 22:54:14.678941 21046 fs_manager.cc:359] Metadata directory not provided
I20260806 22:54:14.678987 21046 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:54:14.679176 21046 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:54:14.680178 21046 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056586788339-29547-0/minicluster-data/master-0/data/instance:
uuid: "b8978bd1c2254a8b80ba736378625821"
format_stamp: "Formatted at 2026-08-06 22:54:14 on dist-test-slave-1xrh"
server_key: "7f202fdacc2145a59016437f196c742e"
server_key_iv: "027e60b35231701873ac4aff59b511b9"
server_key_version: "encryptionkey@0"
I20260806 22:54:14.680531 21046 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056586788339-29547-0/minicluster-data/master-0/wal/instance:
uuid: "b8978bd1c2254a8b80ba736378625821"
format_stamp: "Formatted at 2026-08-06 22:54:14 on dist-test-slave-1xrh"
server_key: "7f202fdacc2145a59016437f196c742e"
server_key_iv: "027e60b35231701873ac4aff59b511b9"
server_key_version: "encryptionkey@0"
I20260806 22:54:14.681804 21046 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 22:54:14.682544 21061 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:54:14.682720 21046 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 22:54:14.682812 21046 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056586788339-29547-0/minicluster-data/master-0/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056586788339-29547-0/minicluster-data/master-0/wal
uuid: "b8978bd1c2254a8b80ba736378625821"
format_stamp: "Formatted at 2026-08-06 22:54:14 on dist-test-slave-1xrh"
server_key: "7f202fdacc2145a59016437f196c742e"
server_key_iv: "027e60b35231701873ac4aff59b511b9"
server_key_version: "encryptionkey@0"
I20260806 22:54:14.682888 21046 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056586788339-29547-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056586788339-29547-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056586788339-29547-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:54:14.699395 21046 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:54:14.701261 21046 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:54:14.701426 21046 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:54:14.705631 21046 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.254:34875
I20260806 22:54:14.705667 21118 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.254:34875 every 8 connection(s)
I20260806 22:54:14.706009 21046 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056586788339-29547-0/minicluster-data/master-0/data/info.pb
I20260806 22:54:14.706549 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 21046
I20260806 22:54:14.706664 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056586788339-29547-0/minicluster-data/master-0/wal/instance
I20260806 22:54:14.706881 29547 external_mini_cluster.cc:1428] Setting key 550a05f0e60b6f8fba3c695533465e04
I20260806 22:54:14.707077 21119 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:54:14.709286 21119 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.218.254" port: 34875 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.28.218.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.218.253@KRBTEST.COM" created.
Aug 06 22:54:14 dist-test-slave-1xrh krb5kdc[21033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056854, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.254@KRBTEST.COM for kudu/127.28.218.254@KRBTEST.COM
I20260806 22:54:14.722313 21123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:54:14.710090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53064 (local address 127.28.218.254:34875)
0806 22:54:14.710251 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0806 22:54:14.710255 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:54:14.710297 (+ 42us) server_negotiation.cc:407] Connection header received
0806 22:54:14.710718 (+ 421us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:54:14.710721 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:54:14.710744 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:54:14.710843 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:54:14.711452 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:54:14.712799 (+ 1347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:54:14.713641 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:54:14.714158 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:54:14.714271 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:54:14.718967 (+ 4696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:54:14.718976 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:54:14.718976 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:54:14.718984 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:54:14.720695 (+ 1711us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:54:14.721120 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:54:14.721123 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:54:14.721123 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:54:14.721212 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:54:14.721462 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:54:14.721463 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:54:14.721464 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:54:14.721671 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:54:14.721763 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0806 22:54:14.721941 (+ 178us) server_negotiation.cc:299] Negotiation successful
0806 22:54:14.722078 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":68,"threads_started":1}
I20260806 22:54:14.722332 21122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:54:14.709897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:34875 (local address 127.0.0.1:53064)
0806 22:54:14.710054 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0806 22:54:14.710094 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 22:54:14.710553 (+ 459us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:54:14.710879 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:54:14.710881 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:54:14.711209 (+ 328us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:54:14.711352 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:54:14.711356 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:54:14.712894 (+ 1538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:54:14.712895 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:54:14.713553 (+ 658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:54:14.713554 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:54:14.714256 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:54:14.714257 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:54:14.714318 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:54:14.714819 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:54:14.714826 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:54:14.718881 (+ 4055us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:54:14.720780 (+ 1899us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:54:14.720788 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:54:14.720789 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:54:14.720997 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:54:14.721282 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:54:14.721283 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:54:14.721284 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:54:14.721385 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:54:14.721742 (+ 357us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:54:14.721747 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:54:14.721808 (+ 61us) client_negotiation.cc:769] Sending connection context
0806 22:54:14.721907 (+ 99us) client_negotiation.cc:240] Negotiation successful
0806 22:54:14.722142 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"mutex_wait_us":96,"thread_start_us":88,"threads_started":1}
I20260806 22:54:14.722991 21119 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.218.253" port: 45491 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:54:14.723376 21122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:54:14.723255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.253:45491 (local address 127.0.0.1:56544)
0806 22:54:14.723299 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0806 22:54:14.723350 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.253:45491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260806 22:54:14.723492 21066 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.28.218.253:45491: connect: Connection refused (error 111)
I20260806 22:54:14.723939 21122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:54:14.723847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.253:45491 (local address 127.0.0.1:56558)
0806 22:54:14.723892 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0806 22:54:14.723916 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.253:45491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260806 22:54:14.724043 21119 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.218.253:45491: Network error: Client connection negotiation failed: client connection to 127.28.218.253:45491: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.218.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.253.keytab.
Entry for principal kudu/127.28.218.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.253.keytab.
I20260806 22:54:14.733590 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056586788339-29547-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056586788339-29547-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056586788339-29547-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056586788339-29547-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.218.253:45491
--webserver_interface=127.28.218.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.253.keytab
--principal=kudu/127.28.218.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.218.212:41199
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.218.254:34875,127.28.218.253:45491,127.28.218.252:40877
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:54:14.773265 21119 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.218.253" port: 45491 } attempt: 1
I20260806 22:54:14.773803 21122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:54:14.773671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.253:45491 (local address 127.0.0.1:56566)
0806 22:54:14.773737 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0806 22:54:14.773773 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.253:45491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260806 22:54:14.774168 21122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:54:14.774073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.253:45491 (local address 127.0.0.1:56574)
0806 22:54:14.774117 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0806 22:54:14.774147 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.253:45491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260806 22:54:14.774257 21119 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.218.253:45491: Network error: Client connection negotiation failed: client connection to 127.28.218.253:45491: connect: Connection refused (error 111)
W20260806 22:54:14.812960 21127 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 22:54:14.813184 21127 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:54:14.813213 21127 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:54:14.813252 21127 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:54:14.814380 21127 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:54:14.814448 21127 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:54:14.814476 21127 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:54:14.814507 21127 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:54:14.814530 21127 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:54:14.816006 21127 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:41199
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056586788339-29547-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056586788339-29547-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.28.218.254:34875,127.28.218.253:45491,127.28.218.252:40877
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.253.keytab
--principal=kudu/127.28.218.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.218.253:45491
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056586788339-29547-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.28.218.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21127
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056586788339-29547-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:54:14.816268 21127 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:54:14.816480 21127 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:54:14.822183 21133 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:54:14.822218 21132 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:54:14.822357 21127 server_base.cc:1034] running on GCE node
W20260806 22:54:14.822190 21135 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:54:14.822675 21127 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:54:14.822954 21127 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:54:14.824088 21127 hybrid_clock.cc:648] HybridClock initialized: now 1786056854824060 us; error 46 us; skew 500 ppm
Aug 06 22:54:14 dist-test-slave-1xrh krb5kdc[21033](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056854, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:54:14.826591 21127 init.cc:377] Logged in from keytab as kudu/127.28.218.253@KRBTEST.COM (short username kudu)
I20260806 22:54:14.827013 21127 webserver.cc:460] Webserver started at http://127.28.218.253:35539/ using document root <none> and password file <none>
I20260806 22:54:14.827250 21127 fs_manager.cc:359] Metadata directory not provided
I20260806 22:54:14.827296 21127 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:54:14.827445 21127 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:54:14.828521 21127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056586788339-29547-0/minicluster-data/master-1/data/instance:
uuid: "720d2c909a504af894246cd0d7986856"
format_stamp: "Formatted at 2026-08-06 22:54:14 on dist-test-slave-1xrh"
server_key: "ae0d5ff7d09d3914dca960a340af030e"
server_key_iv: "35061a15ea8052d9499b191ea79d15c1"
server_key_version: "encryptionkey@0"
I20260806 22:54:14.828864 21127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056586788339-29547-0/minicluster-data/master-1/wal/instance:
uuid: "720d2c909a504af894246cd0d7986856"
format_stamp: "Formatted at 2026-08-06 22:54:14 on dist-test-slave-1xrh"
server_key: "ae0d5ff7d09d3914dca960a340af030e"
server_key_iv: "35061a15ea8052d9499b191ea79d15c1"
server_key_version: "encryptionkey@0"
I20260806 22:54:14.830163 21127 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:54:14.830927 21142 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:54:14.831176 21127 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:54:14.831269 21127 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056586788339-29547-0/minicluster-data/master-1/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056586788339-29547-0/minicluster-data/master-1/wal
uuid: "720d2c909a504af894246cd0d7986856"
format_stamp: "Formatted at 2026-08-06 22:54:14 on dist-test-slave-1xrh"
server_key: "ae0d5ff7d09d3914dca960a340af030e"
server_key_iv: "35061a15ea8052d9499b191ea79d15c1"
server_key_version: "encryptionkey@0"
I20260806 22:54:14.831338 21127 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056586788339-29547-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056586788339-29547-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056586788339-29547-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:54:14.840674 21127 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:54:14.842270 21127 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:54:14.842406 21127 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:54:14.842389 21119 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.218.253" port: 45491 } attempt: 2
I20260806 22:54:14.842813 21122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:54:14.842654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.253:45491 (local address 127.0.0.1:56588)
0806 22:54:14.842716 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0806 22:54:14.842759 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.253:45491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260806 22:54:14.843237 21122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:54:14.843099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.253:45491 (local address 127.0.0.1:56600)
0806 22:54:14.843184 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0806 22:54:14.843213 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.253:45491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
W20260806 22:54:14.843353 21119 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.218.253:45491: Network error: Client connection negotiation failed: client connection to 127.28.218.253:45491: connect: Connection refused (error 111)
I20260806 22:54:14.846369 21127 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.253:45491
I20260806 22:54:14.846413 21198 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.253:45491 every 8 connection(s)
I20260806 22:54:14.846719 21127 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056586788339-29547-0/minicluster-data/master-1/data/info.pb
I20260806 22:54:14.847628 21199 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:54:14.848111 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 21127
I20260806 22:54:14.848223 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056586788339-29547-0/minicluster-data/master-1/wal/instance
I20260806 22:54:14.848410 29547 external_mini_cluster.cc:1428] Setting key 842775ddfab7133ef6834a896a852924
I20260806 22:54:14.849242 21199 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.218.254" port: 34875 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 22:54:14 dist-test-slave-1xrh krb5kdc[21033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056854, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.253@KRBTEST.COM for kudu/127.28.218.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.28.218.252@KRBTEST.COM; defaulting to no policy
I20260806 22:54:14.858146 21123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:54:14.849885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53066 (local address 127.28.218.254:34875)
0806 22:54:14.849939 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0806 22:54:14.849941 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:54:14.850132 (+ 191us) server_negotiation.cc:407] Connection header received
0806 22:54:14.850643 (+ 511us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:54:14.850644 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:54:14.850655 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:54:14.850712 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:54:14.851356 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:54:14.851815 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:54:14.852754 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:54:14.852953 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:54:14.853026 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:54:14.855407 (+ 2381us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:54:14.855408 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:54:14.855408 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:54:14.855415 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:54:14.856862 (+ 1447us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:54:14.857225 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:54:14.857226 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:54:14.857227 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:54:14.857273 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:54:14.857604 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:54:14.857605 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:54:14.857606 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:54:14.857719 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:54:14.857782 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 22:54:14.858026 (+ 244us) server_negotiation.cc:299] Negotiation successful
0806 22:54:14.858092 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260806 22:54:14.858212 21203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:54:14.849874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:34875 (local address 127.0.0.1:53066)
0806 22:54:14.850024 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0806 22:54:14.850057 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0806 22:54:14.850543 (+ 486us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:54:14.850779 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:54:14.850782 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:54:14.851015 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:54:14.851251 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:54:14.851255 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:54:14.851906 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:54:14.851908 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:54:14.852670 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:54:14.852672 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:54:14.853037 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:54:14.853039 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:54:14.853150 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:54:14.853628 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:54:14.853636 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:54:14.855316 (+ 1680us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:54:14.856932 (+ 1616us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:54:14.856939 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:54:14.856940 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:54:14.857151 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:54:14.857339 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:54:14.857340 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:54:14.857340 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:54:14.857507 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:54:14.857780 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:54:14.857786 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:54:14.857892 (+ 106us) client_negotiation.cc:769] Sending connection context
0806 22:54:14.857970 (+ 78us) client_negotiation.cc:240] Negotiation successful
0806 22:54:14.858074 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":85,"threads_started":1}
I20260806 22:54:14.858819 21199 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.218.253" port: 45491 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.218.252@KRBTEST.COM" created.
Aug 06 22:54:14 dist-test-slave-1xrh krb5kdc[21033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056854, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.253@KRBTEST.COM for kudu/127.28.218.253@KRBTEST.COM
I20260806 22:54:14.865717 21203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:54:14.859093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.253:45491 (local address 127.0.0.1:56604)
0806 22:54:14.859159 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0806 22:54:14.859170 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:54:14.859212 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:54:14.859391 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:54:14.859392 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:54:14.859546 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:54:14.859603 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:54:14.859604 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:54:14.860396 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:54:14.860397 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:54:14.860855 (+ 458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:54:14.860856 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:54:14.861286 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:54:14.861287 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:54:14.861344 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:54:14.861679 (+ 335us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:54:14.861684 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:54:14.862967 (+ 1283us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:54:14.864718 (+ 1751us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:54:14.864718 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:54:14.864719 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:54:14.864887 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:54:14.865094 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:54:14.865094 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:54:14.865095 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:54:14.865143 (+ 48us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:54:14.865513 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:54:14.865514 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:54:14.865548 (+ 34us) client_negotiation.cc:769] Sending connection context
0806 22:54:14.865603 (+ 55us) client_negotiation.cc:240] Negotiation successful
0806 22:54:14.865664 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260806 22:54:14.865876 21204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:54:14.859106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56604 (local address 127.28.218.253:45491)
0806 22:54:14.859244 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0806 22:54:14.859248 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:54:14.859266 (+ 18us) server_negotiation.cc:407] Connection header received
0806 22:54:14.859302 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:54:14.859304 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:54:14.859318 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:54:14.859384 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:54:14.859669 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:54:14.860297 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:54:14.860941 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:54:14.861208 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:54:14.861288 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:54:14.863056 (+ 1768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:54:14.863063 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:54:14.863063 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:54:14.863069 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:54:14.864619 (+ 1550us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:54:14.864969 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:54:14.864972 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:54:14.864972 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:54:14.865036 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:54:14.865249 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:54:14.865250 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:54:14.865250 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:54:14.865426 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:54:14.865536 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 22:54:14.865653 (+ 117us) server_negotiation.cc:299] Negotiation successful
0806 22:54:14.865737 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":91,"threads_started":1}
I20260806 22:54:14.866636 21199 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.218.252" port: 40877 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:54:14.866935 21203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:54:14.866820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.252:40877 (local address 127.0.0.1:52966)
0806 22:54:14.866869 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0806 22:54:14.866912 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.252:40877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260806 22:54:14.867038 21146 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.28.218.252:40877: connect: Connection refused (error 111)
I20260806 22:54:14.867463 21203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:54:14.867372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.252:40877 (local address 127.0.0.1:52978)
0806 22:54:14.867411 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0806 22:54:14.867435 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.252:40877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260806 22:54:14.867657 21199 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.218.252:40877: Network error: Client connection negotiation failed: client connection to 127.28.218.252:40877: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.218.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.252.keytab.
Entry for principal kudu/127.28.218.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.252.keytab.
I20260806 22:54:14.873895 29547 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQja9Us/build/release/bin/kudu
/tmp/dist-test-taskQja9Us/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056586788339-29547-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056586788339-29547-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056586788339-29547-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056586788339-29547-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.218.252:40877
--webserver_interface=127.28.218.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.252.keytab
--principal=kudu/127.28.218.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.218.212:41199
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.218.254:34875,127.28.218.253:45491,127.28.218.252:40877
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056586788339-29547-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056586788339-29547-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056586788339-29547-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:54:14.916862 21199 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.218.252" port: 40877 } attempt: 1
I20260806 22:54:14.917353 21203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:54:14.917202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.252:40877 (local address 127.0.0.1:52992)
0806 22:54:14.917270 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0806 22:54:14.917316 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.252:40877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260806 22:54:14.917811 21203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:54:14.917722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.252:40877 (local address 127.0.0.1:52994)
0806 22:54:14.917764 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0806 22:54:14.917792 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.252:40877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260806 22:54:14.917984 21199 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.218.252:40877: Network error: Client connection negotiation failed: client connection to 127.28.218.252:40877: connect: Connection refused (error 111)
I20260806 22:54:14.934486 21119 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.218.253" port: 45491 } attempt: 3
Aug 06 22:54:14 dist-test-slave-1xrh krb5kdc[21033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056854, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.254@KRBTEST.COM for kudu/127.28.218.253@KRBTEST.COM
I20260806 22:54:14.942219 21122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:54:14.934828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.253:45491 (local address 127.0.0.1:56618)
0806 22:54:14.934898 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0806 22:54:14.934914 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 22:54:14.934983 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:54:14.935185 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:54:14.935186 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:54:14.935413 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:54:14.935516 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:54:14.935519 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:54:14.936208 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:54:14.936210 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:54:14.937084 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:54:14.937087 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:54:14.937485 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:54:14.937486 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:54:14.937568 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:54:14.938140 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:54:14.938146 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:54:14.939584 (+ 1438us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:54:14.941238 (+ 1654us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:54:14.941239 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:54:14.941239 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:54:14.941410 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:54:14.941634 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:54:14.941635 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:54:14.941635 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:54:14.941687 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:54:14.941975 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:54:14.941977 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:54:14.942029 (+ 52us) client_negotiation.cc:769] Sending connection context
0806 22:54:14.942083 (+ 54us) client_negotiation.cc:240] Negotiation successful
0806 22:54:14.942155 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260806 22:54:14.942291 21204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:54:14.934928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56618 (local address 127.28.218.253:45491)
0806 22:54:14.934982 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0806 22:54:14.934984 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:54:14.935000 (+ 16us) server_negotiation.cc:407] Connection header received
0806 22:54:14.935050 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:54:14.935051 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:54:14.935059 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:54:14.935172 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:54:14.935619 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:54:14.936123 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:54:14.937188 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:54:14.937414 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:54:14.937467 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:54:14.939669 (+ 2202us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:54:14.939670 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:54:14.939670 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:54:14.939679 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:54:14.941148 (+ 1469us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:54:14.941492 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:54:14.941493 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:54:14.941494 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:54:14.941558 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:54:14.941742 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:54:14.941743 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:54:14.941743 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:54:14.941886 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:54:14.941951 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 22:54:14.942130 (+ 179us) server_negotiation.cc:299] Negotiation successful
0806 22:54:14.942209 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260806 22:54:14.942622 21119 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.218.252" port: 40877 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:54:14.942971 21122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:54:14.942834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.252:40877 (local address 127.0.0.1:53008)
0806 22:54:14.942888 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0806 22:54:14.942941 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.252:40877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260806 22:54:14.943341 21122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:54:14.943243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.252:40877 (local address 127.0.0.1:53012)
0806 22:54:14.943287 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0806 22:54:14.943321 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.252:40877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260806 22:54:14.943501 21119 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.218.252:40877: Network error: Client connection negotiation failed: client connection to 127.28.218.252:40877: connect: Connection refused (error 111)
W20260806 22:54:14.955272 21207 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 22:54:14.955464 21207 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:54:14.955510 21207 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:54:14.955549 21207 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:54:14.956631 21207 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:54:14.956689 21207 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:54:14.956717 21207 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:54:14.956744 21207 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:54:14.956768 21207 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:54:14.958195 21207 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.218.212:41199
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056586788339-29547-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056586788339-29547-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.28.218.254:34875,127.28.218.253:45491,127.28.218.252:40877
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056586788339-29547-0/krb5kdc/kudu_127.28.218.252.keytab
--principal=kudu/127.28.218.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.218.252:40877
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056586788339-29547-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.28.218.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21207
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056586788339-29547-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:54:14.958451 21207 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:54:14.958652 21207 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:54:14.963183 21213 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:54:14.963192 21215 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:54:14.963263 21212 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:54:14.963395 21207 server_base.cc:1034] running on GCE node
I20260806 22:54:14.963644 21207 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:54:14.963922 21207 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:54:14.965085 21207 hybrid_clock.cc:648] HybridClock initialized: now 1786056854965060 us; error 35 us; skew 500 ppm
Aug 06 22:54:14 dist-test-slave-1xrh krb5kdc[21033](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056854, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:54:14.967504 21207 init.cc:377] Logged in from keytab as kudu/127.28.218.252@KRBTEST.COM (short username kudu)
I20260806 22:54:14.967947 21207 webserver.cc:460] Webserver started at http://127.28.218.252:44177/ using document root <none> and password file <none>
I20260806 22:54:14.968169 21207 fs_manager.cc:359] Metadata directory not provided
I20260806 22:54:14.968230 21207 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:54:14.968359 21207 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:54:14.969297 21207 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056586788339-29547-0/minicluster-data/master-2/data/instance:
uuid: "78ca88aa90374be9b1c08d7bd455fb07"
format_stamp: "Formatted at 2026-08-06 22:54:14 on dist-test-slave-1xrh"
server_key: "2024b330a6b3b5df4d7471a12f2e8cb6"
server_key_iv: "25fa0edcf7405154514a87b6af3400be"
server_key_version: "encryptionkey@0"
I20260806 22:54:14.969667 21207 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056586788339-29547-0/minicluster-data/master-2/wal/instance:
uuid: "78ca88aa90374be9b1c08d7bd455fb07"
format_stamp: "Formatted at 2026-08-06 22:54:14 on dist-test-slave-1xrh"
server_key: "2024b330a6b3b5df4d7471a12f2e8cb6"
server_key_iv: "25fa0edcf7405154514a87b6af3400be"
server_key_version: "encryptionkey@0"
I20260806 22:54:14.970856 21207 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 22:54:14.971571 21223 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:54:14.971731 21207 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 22:54:14.971848 21207 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056586788339-29547-0/minicluster-data/master-2/data,/tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056586788339-29547-0/minicluster-data/master-2/wal
uuid: "78ca88aa90374be9b1c08d7bd455fb07"
format_stamp: "Formatted at 2026-08-06 22:54:14 on dist-test-slave-1xrh"
server_key: "2024b330a6b3b5df4d7471a12f2e8cb6"
server_key_iv: "25fa0edcf7405154514a87b6af3400be"
server_key_version: "encryptionkey@0"
I20260806 22:54:14.971925 21207 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056586788339-29547-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056586788339-29547-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056586788339-29547-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:54:14.974695 21119 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.218.252" port: 40877 } attempt: 1
I20260806 22:54:14.975059 21122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:54:14.974931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.252:40877 (local address 127.0.0.1:53020)
0806 22:54:14.974979 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0806 22:54:14.975014 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.252:40877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260806 22:54:14.975436 21122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:54:14.975330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.252:40877 (local address 127.0.0.1:53030)
0806 22:54:14.975374 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0806 22:54:14.975402 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.252:40877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260806 22:54:14.975543 21119 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.218.252:40877: Network error: Client connection negotiation failed: client connection to 127.28.218.252:40877: connect: Connection refused (error 111)
I20260806 22:54:14.986150 21199 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.218.252" port: 40877 } attempt: 2
I20260806 22:54:14.986479 21203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:54:14.986372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.252:40877 (local address 127.0.0.1:53038)
0806 22:54:14.986423 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0806 22:54:14.986454 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.252:40877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260806 22:54:14.986886 21203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:54:14.986771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.252:40877 (local address 127.0.0.1:53040)
0806 22:54:14.986805 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0806 22:54:14.986848 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.218.252:40877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
W20260806 22:54:14.987004 21199 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.218.252:40877: Network error: Client connection negotiation failed: client connection to 127.28.218.252:40877: connect: Connection refused (error 111)
I20260806 22:54:14.988010 21207 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:54:14.989919 21207 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:54:14.990082 21207 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:54:14.993703 21207 rpc_server.cc:307] RPC server started. Bound to: 127.28.218.252:40877
I20260806 22:54:14.993777 21278 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.218.252:40877 every 8 connection(s)
I20260806 22:54:14.994088 21207 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056586788339-29547-0/minicluster-data/master-2/data/info.pb
I20260806 22:54:14.995016 21279 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:54:14.996479 21279 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.218.254" port: 34875 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:54:14.998425 29547 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQja9Us/build/release/bin/kudu as pid 21207
I20260806 22:54:14.998533 29547 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQja9Us/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056586788339-29547-0/minicluster-data/master-2/wal/instance
I20260806 22:54:14.998749 29547 external_mini_cluster.cc:1428] Setting key 0a0e991a8c999ff5675e5b8b0504a69c
Aug 06 22:54:14 dist-test-slave-1xrh krb5kdc[21033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056854, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.252@KRBTEST.COM for kudu/127.28.218.254@KRBTEST.COM
I20260806 22:54:15.005889 21123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:54:14.997122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53072 (local address 127.28.218.254:34875)
0806 22:54:14.997181 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0806 22:54:14.997183 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:54:14.997352 (+ 169us) server_negotiation.cc:407] Connection header received
0806 22:54:14.997802 (+ 450us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:54:14.997803 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:54:14.997812 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:54:14.997887 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:54:14.998401 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:54:14.999157 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:54:15.000140 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:54:15.000333 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:54:15.000404 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:54:15.002740 (+ 2336us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:54:15.002742 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:54:15.002742 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:54:15.002751 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:54:15.004359 (+ 1608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:54:15.004828 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:54:15.004830 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:54:15.004830 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:54:15.004892 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:54:15.005129 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:54:15.005130 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:54:15.005131 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:54:15.005288 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:54:15.005372 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 22:54:15.005642 (+ 270us) server_negotiation.cc:299] Negotiation successful
0806 22:54:15.005737 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 22:54:15.005983 21281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:54:14.997099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.254:34875 (local address 127.0.0.1:53072)
0806 22:54:14.997236 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0806 22:54:14.997275 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 22:54:14.997712 (+ 437us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:54:14.997933 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:54:14.997936 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:54:14.998147 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:54:14.998318 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:54:14.998321 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:54:14.999261 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:54:14.999262 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:54:15.000052 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:54:15.000055 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:54:15.000409 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:54:15.000409 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:54:15.000545 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:54:15.001101 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:54:15.001114 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:54:15.002641 (+ 1527us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:54:15.004443 (+ 1802us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:54:15.004449 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:54:15.004451 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:54:15.004735 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:54:15.004972 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:54:15.004973 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:54:15.004973 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:54:15.005066 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:54:15.005384 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:54:15.005390 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:54:15.005500 (+ 110us) client_negotiation.cc:769] Sending connection context
0806 22:54:15.005595 (+ 95us) client_negotiation.cc:240] Negotiation successful
0806 22:54:15.005701 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":70,"threads_started":1}
Aug 06 22:54:15 dist-test-slave-1xrh krb5kdc[21033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056854, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.218.252@KRBTEST.COM
I20260806 22:54:15.006582 21279 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.218.253" port: 45491 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:54:15.009868 21285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:54:14.999090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53052 (local address 127.28.218.252:40877)
0806 22:54:15.000166 (+ 1076us) server_negotiation.cc:206] Beginning negotiation
0806 22:54:15.000168 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:54:15.000180 (+ 12us) server_negotiation.cc:407] Connection header received
0806 22:54:15.000208 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:54:15.000209 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:54:15.000223 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:54:15.000300 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:54:15.001632 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:54:15.002664 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:54:15.003818 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:54:15.004099 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:54:15.004156 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:54:15.007113 (+ 2957us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:54:15.007140 (+ 27us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:54:15.007141 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:54:15.007160 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:54:15.008624 (+ 1464us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:54:15.009049 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:54:15.009052 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:54:15.009053 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:54:15.009129 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:54:15.009335 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:54:15.009336 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:54:15.009336 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:54:15.009491 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:54:15.009665 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0806 22:54:15.009709 (+ 44us) server_negotiation.cc:299] Negotiation successful
0806 22:54:15.009777 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1055,"thread_start_us":94,"threads_started":1}
Aug 06 22:54:15 dist-test-slave-1xrh krb5kdc[21033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056854, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.252@KRBTEST.COM for kudu/127.28.218.253@KRBTEST.COM
I20260806 22:54:15.013103 21281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:54:15.006853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.253:45491 (local address 127.0.0.1:56628)
0806 22:54:15.006885 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0806 22:54:15.006895 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:54:15.006936 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:54:15.007051 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:54:15.007052 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:54:15.007223 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:54:15.007267 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:54:15.007268 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:54:15.007869 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:54:15.007870 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:54:15.008322 (+ 452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:54:15.008323 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:54:15.008689 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:54:15.008689 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:54:15.008754 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:54:15.009148 (+ 394us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:54:15.009153 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:54:15.010510 (+ 1357us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:54:15.012152 (+ 1642us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:54:15.012153 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:54:15.012153 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:54:15.012370 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:54:15.012533 (+ 163us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:54:15.012534 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:54:15.012534 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:54:15.012590 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:54:15.012797 (+ 207us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:54:15.012798 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:54:15.012840 (+ 42us) client_negotiation.cc:769] Sending connection context
0806 22:54:15.012925 (+ 85us) client_negotiation.cc:240] Negotiation successful
0806 22:54:15.013016 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16}
I20260806 22:54:15.013124 21204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:54:15.006884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56628 (local address 127.28.218.253:45491)
0806 22:54:15.006933 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 22:54:15.006935 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:54:15.006944 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:54:15.006993 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:54:15.006994 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:54:15.007003 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:54:15.007050 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:54:15.007365 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:54:15.007789 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:54:15.008413 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:54:15.008622 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:54:15.008691 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:54:15.010596 (+ 1905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:54:15.010597 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:54:15.010598 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:54:15.010606 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:54:15.012070 (+ 1464us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:54:15.012436 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:54:15.012436 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:54:15.012437 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:54:15.012484 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:54:15.012642 (+ 158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:54:15.012642 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:54:15.012643 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:54:15.012728 (+ 85us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:54:15.012805 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 22:54:15.012961 (+ 156us) server_negotiation.cc:299] Negotiation successful
0806 22:54:15.013033 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 22:54:15.013664 21279 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.218.252" port: 40877 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 22:54:15 dist-test-slave-1xrh krb5kdc[21033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056854, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.218.254@KRBTEST.COM
Aug 06 22:54:15 dist-test-slave-1xrh krb5kdc[21033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056854, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.252@KRBTEST.COM for kudu/127.28.218.252@KRBTEST.COM
I20260806 22:54:15.018210 21123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:54:15.010759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53082 (local address 127.28.218.254:34875)
0806 22:54:15.010791 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0806 22:54:15.010793 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:54:15.010802 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:54:15.010826 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:54:15.010827 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:54:15.010835 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:54:15.010896 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:54:15.011399 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:54:15.011823 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:54:15.012737 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:54:15.012935 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:54:15.013004 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:54:15.015496 (+ 2492us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:54:15.015497 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:54:15.015498 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:54:15.015504 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:54:15.017113 (+ 1609us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:54:15.017418 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:54:15.017419 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:54:15.017419 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:54:15.017476 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:54:15.017653 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:54:15.017654 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:54:15.017654 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:54:15.017787 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:54:15.017976 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0806 22:54:15.018017 (+ 41us) server_negotiation.cc:299] Negotiation successful
0806 22:54:15.018100 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260806 22:54:15.020843 21281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:54:15.013891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.252:40877 (local address 127.0.0.1:53066)
0806 22:54:15.013943 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0806 22:54:15.013955 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:54:15.014016 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:54:15.014128 (+ 112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:54:15.014129 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:54:15.014264 (+ 135us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:54:15.014302 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:54:15.014303 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:54:15.015220 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:54:15.015221 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:54:15.015813 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:54:15.015816 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:54:15.016294 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:54:15.016295 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:54:15.016345 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:54:15.016720 (+ 375us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:54:15.016723 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:54:15.017962 (+ 1239us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:54:15.019753 (+ 1791us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:54:15.019754 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:54:15.019755 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:54:15.019986 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:54:15.020187 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:54:15.020187 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:54:15.020187 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:54:15.020306 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:54:15.020549 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:54:15.020549 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:54:15.020581 (+ 32us) client_negotiation.cc:769] Sending connection context
0806 22:54:15.020630 (+ 49us) client_negotiation.cc:240] Negotiation successful
0806 22:54:15.020701 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260806 22:54:15.020906 21285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:54:15.013921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53066 (local address 127.28.218.252:40877)
0806 22:54:15.013960 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0806 22:54:15.013962 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:54:15.014004 (+ 42us) server_negotiation.cc:407] Connection header received
0806 22:54:15.014075 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:54:15.014076 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:54:15.014082 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:54:15.014123 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:54:15.014380 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:54:15.015105 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:54:15.015930 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:54:15.016200 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:54:15.016293 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:54:15.018044 (+ 1751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:54:15.018045 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:54:15.018045 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:54:15.018052 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:54:15.019678 (+ 1626us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:54:15.020057 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:54:15.020058 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:54:15.020058 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:54:15.020122 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:54:15.020368 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:54:15.020369 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:54:15.020369 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:54:15.020486 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:54:15.020550 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0806 22:54:15.020651 (+ 101us) server_negotiation.cc:299] Negotiation successful
0806 22:54:15.020771 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":35,"server-negotiator.queue_time_us":14}
I20260806 22:54:15.022384 21279 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 78ca88aa90374be9b1c08d7bd455fb07: Bootstrap starting.
I20260806 22:54:15.023491 21279 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 78ca88aa90374be9b1c08d7bd455fb07: Neither blocks nor log segments found. Creating new log.
I20260806 22:54:15.023775 21279 log.cc:824] T 00000000000000000000000000000000 P 78ca88aa90374be9b1c08d7bd455fb07: Log is configured to *not* fsync() on all Append() calls
Aug 06 22:54:15 dist-test-slave-1xrh krb5kdc[21033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056854, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.218.253@KRBTEST.COM
I20260806 22:54:15.024601 21279 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 78ca88aa90374be9b1c08d7bd455fb07: No bootstrap required, opened a new log
I20260806 22:54:15.026954 21279 raft_consensus.cc:348] T 00000000000000000000000000000000 P 78ca88aa90374be9b1c08d7bd455fb07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8978bd1c2254a8b80ba736378625821" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 34875 } } peers { permanent_uuid: "720d2c909a504af894246cd0d7986856" member_type: VOTER last_known_addr { host: "127.28.218.253" port: 45491 } } peers { permanent_uuid: "78ca88aa90374be9b1c08d7bd455fb07" member_type: VOTER last_known_addr { host: "127.28.218.252" port: 40877 } }
I20260806 22:54:15.027114 21279 raft_consensus.cc:729] T 00000000000000000000000000000000 P 78ca88aa90374be9b1c08d7bd455fb07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78ca88aa90374be9b1c08d7bd455fb07, State: Initialized, Role: FOLLOWER
I20260806 22:54:15.027273 21279 consensus_queue.cc:260] T 00000000000000000000000000000000 P 78ca88aa90374be9b1c08d7bd455fb07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8978bd1c2254a8b80ba736378625821" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 34875 } } peers { permanent_uuid: "720d2c909a504af894246cd0d7986856" member_type: VOTER last_known_addr { host: "127.28.218.253" port: 45491 } } peers { permanent_uuid: "78ca88aa90374be9b1c08d7bd455fb07" member_type: VOTER last_known_addr { host: "127.28.218.252" port: 40877 } }
I20260806 22:54:15.027410 21204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:54:15.019330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56630 (local address 127.28.218.253:45491)
0806 22:54:15.019371 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0806 22:54:15.019373 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:54:15.019381 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:54:15.019401 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:54:15.019401 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:54:15.019409 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:54:15.019457 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:54:15.020030 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:54:15.020677 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:54:15.021654 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:54:15.021883 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:54:15.021937 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:54:15.024449 (+ 2512us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:54:15.024450 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:54:15.024450 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:54:15.024457 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:54:15.025996 (+ 1539us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:54:15.026533 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:54:15.026534 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:54:15.026535 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:54:15.026625 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:54:15.026825 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:54:15.026826 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:54:15.026826 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:54:15.026955 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:54:15.027173 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0806 22:54:15.027216 (+ 43us) server_negotiation.cc:299] Negotiation successful
0806 22:54:15.027293 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 22:54:15.027607 21279 sys_catalog.cc:564] T 00000000000000000000000000000000 P 78ca88aa90374be9b1c08d7bd455fb07 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:54:15.027642 21287 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78ca88aa90374be9b1c08d7bd455fb07 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8978bd1c2254a8b80ba736378625821" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 34875 } } peers { permanent_uuid: "720d2c909a504af894246cd0d7986856" member_type: VOTER last_known_addr { host: "127.28.218.253" port: 45491 } } peers { permanent_uuid: "78ca88aa90374be9b1c08d7bd455fb07" member_type: VOTER last_known_addr { host: "127.28.218.252" port: 40877 } } }
I20260806 22:54:15.027757 21287 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78ca88aa90374be9b1c08d7bd455fb07 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:54:15.028879 29547 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260806 22:54:15.029523 21298 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 78ca88aa90374be9b1c08d7bd455fb07: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 22:54:15.029583 21298 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 22:54:15.029837 21092 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "b8978bd1c2254a8b80ba736378625821"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53082
I20260806 22:54:15.029928 21207 master_runner.cc:186] Error getting master registration for 127.28.218.254:34875: OK, instance_id { permanent_uuid: "b8978bd1c2254a8b80ba736378625821" instance_seqno: 1786056854704465 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 22:54:15.032008 21207 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 22:54:15.050680 21119 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.218.252" port: 40877 } attempt: 2
Aug 06 22:54:15 dist-test-slave-1xrh krb5kdc[21033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056854, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.254@KRBTEST.COM for kudu/127.28.218.252@KRBTEST.COM
I20260806 22:54:15.058559 21122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:54:15.050965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.252:40877 (local address 127.0.0.1:53070)
0806 22:54:15.051016 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0806 22:54:15.051029 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:54:15.051077 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:54:15.051280 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:54:15.051281 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:54:15.051467 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:54:15.051514 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:54:15.051516 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:54:15.052397 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:54:15.052398 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:54:15.053176 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:54:15.053178 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:54:15.053688 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:54:15.053689 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:54:15.053741 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:54:15.054237 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:54:15.054242 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:54:15.055722 (+ 1480us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:54:15.057433 (+ 1711us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:54:15.057433 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:54:15.057434 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:54:15.057671 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:54:15.057873 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:54:15.057874 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:54:15.057875 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:54:15.057936 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:54:15.058259 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:54:15.058260 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:54:15.058300 (+ 40us) client_negotiation.cc:769] Sending connection context
0806 22:54:15.058364 (+ 64us) client_negotiation.cc:240] Negotiation successful
0806 22:54:15.058460 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260806 22:54:15.058564 21285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:54:15.051071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53070 (local address 127.28.218.252:40877)
0806 22:54:15.051154 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0806 22:54:15.051156 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:54:15.051164 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:54:15.051187 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:54:15.051188 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:54:15.051198 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:54:15.051266 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:54:15.051606 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:54:15.052323 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:54:15.053263 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:54:15.053601 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:54:15.053668 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:54:15.055795 (+ 2127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:54:15.055797 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:54:15.055797 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:54:15.055804 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:54:15.057328 (+ 1524us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:54:15.057746 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:54:15.057747 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:54:15.057748 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:54:15.057805 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:54:15.058000 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:54:15.058001 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:54:15.058002 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:54:15.058184 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:54:15.058248 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0806 22:54:15.058403 (+ 155us) server_negotiation.cc:299] Negotiation successful
0806 22:54:15.058477 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 22:54:15.060215 21119 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b8978bd1c2254a8b80ba736378625821: Bootstrap starting.
I20260806 22:54:15.060900 21119 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b8978bd1c2254a8b80ba736378625821: Neither blocks nor log segments found. Creating new log.
I20260806 22:54:15.061187 21119 log.cc:824] T 00000000000000000000000000000000 P b8978bd1c2254a8b80ba736378625821: Log is configured to *not* fsync() on all Append() calls
I20260806 22:54:15.062048 21119 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b8978bd1c2254a8b80ba736378625821: No bootstrap required, opened a new log
I20260806 22:54:15.063380 21119 raft_consensus.cc:348] T 00000000000000000000000000000000 P b8978bd1c2254a8b80ba736378625821 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8978bd1c2254a8b80ba736378625821" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 34875 } } peers { permanent_uuid: "720d2c909a504af894246cd0d7986856" member_type: VOTER last_known_addr { host: "127.28.218.253" port: 45491 } } peers { permanent_uuid: "78ca88aa90374be9b1c08d7bd455fb07" member_type: VOTER last_known_addr { host: "127.28.218.252" port: 40877 } }
I20260806 22:54:15.063531 21119 raft_consensus.cc:729] T 00000000000000000000000000000000 P b8978bd1c2254a8b80ba736378625821 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8978bd1c2254a8b80ba736378625821, State: Initialized, Role: FOLLOWER
I20260806 22:54:15.063611 21119 consensus_queue.cc:260] T 00000000000000000000000000000000 P b8978bd1c2254a8b80ba736378625821 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8978bd1c2254a8b80ba736378625821" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 34875 } } peers { permanent_uuid: "720d2c909a504af894246cd0d7986856" member_type: VOTER last_known_addr { host: "127.28.218.253" port: 45491 } } peers { permanent_uuid: "78ca88aa90374be9b1c08d7bd455fb07" member_type: VOTER last_known_addr { host: "127.28.218.252" port: 40877 } }
I20260806 22:54:15.063915 21119 sys_catalog.cc:564] T 00000000000000000000000000000000 P b8978bd1c2254a8b80ba736378625821 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:54:15.063879 21301 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8978bd1c2254a8b80ba736378625821 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8978bd1c2254a8b80ba736378625821" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 34875 } } peers { permanent_uuid: "720d2c909a504af894246cd0d7986856" member_type: VOTER last_known_addr { host: "127.28.218.253" port: 45491 } } peers { permanent_uuid: "78ca88aa90374be9b1c08d7bd455fb07" member_type: VOTER last_known_addr { host: "127.28.218.252" port: 40877 } } }
I20260806 22:54:15.063979 21301 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8978bd1c2254a8b80ba736378625821 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:54:15.063938 21092 raft_consensus.cc:482] T 00000000000000000000000000000000 P b8978bd1c2254a8b80ba736378625821 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260806 22:54:15.064097 21092 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b8978bd1c2254a8b80ba736378625821 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:54:15.064807 21092 raft_consensus.cc:504] T 00000000000000000000000000000000 P b8978bd1c2254a8b80ba736378625821 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8978bd1c2254a8b80ba736378625821" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 34875 } } peers { permanent_uuid: "720d2c909a504af894246cd0d7986856" member_type: VOTER last_known_addr { host: "127.28.218.253" port: 45491 } } peers { permanent_uuid: "78ca88aa90374be9b1c08d7bd455fb07" member_type: VOTER last_known_addr { host: "127.28.218.252" port: 40877 } }
I20260806 22:54:15.065402 21251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b8978bd1c2254a8b80ba736378625821" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "78ca88aa90374be9b1c08d7bd455fb07"
I20260806 22:54:15.065531 21251 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 78ca88aa90374be9b1c08d7bd455fb07 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:54:15.066403 21251 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 78ca88aa90374be9b1c08d7bd455fb07 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8978bd1c2254a8b80ba736378625821 in term 1.
I20260806 22:54:15.066571 21092 leader_election.cc:288] T 00000000000000000000000000000000 P b8978bd1c2254a8b80ba736378625821 [CANDIDATE]: Term 1 election: Requested vote from peers 720d2c909a504af894246cd0d7986856 (127.28.218.253:45491), 78ca88aa90374be9b1c08d7bd455fb07 (127.28.218.252:40877)
I20260806 22:54:15.066629 21066 leader_election.cc:302] T 00000000000000000000000000000000 P b8978bd1c2254a8b80ba736378625821 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 78ca88aa90374be9b1c08d7bd455fb07, b8978bd1c2254a8b80ba736378625821; no voters:
I20260806 22:54:15.066736 21301 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b8978bd1c2254a8b80ba736378625821 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:54:15.066852 21301 raft_consensus.cc:686] T 00000000000000000000000000000000 P b8978bd1c2254a8b80ba736378625821 [term 1 LEADER]: Becoming Leader. State: Replica: b8978bd1c2254a8b80ba736378625821, State: Running, Role: LEADER
I20260806 22:54:15.066926 21301 consensus_queue.cc:237] T 00000000000000000000000000000000 P b8978bd1c2254a8b80ba736378625821 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8978bd1c2254a8b80ba736378625821" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 34875 } } peers { permanent_uuid: "720d2c909a504af894246cd0d7986856" member_type: VOTER last_known_addr { host: "127.28.218.253" port: 45491 } } peers { permanent_uuid: "78ca88aa90374be9b1c08d7bd455fb07" member_type: VOTER last_known_addr { host: "127.28.218.252" port: 40877 } }
I20260806 22:54:15.067346 21301 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8978bd1c2254a8b80ba736378625821 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b8978bd1c2254a8b80ba736378625821. Latest consensus state: current_term: 1 leader_uuid: "b8978bd1c2254a8b80ba736378625821" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8978bd1c2254a8b80ba736378625821" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 34875 } } peers { permanent_uuid: "720d2c909a504af894246cd0d7986856" member_type: VOTER last_known_addr { host: "127.28.218.253" port: 45491 } } peers { permanent_uuid: "78ca88aa90374be9b1c08d7bd455fb07" member_type: VOTER last_known_addr { host: "127.28.218.252" port: 40877 } } }
I20260806 22:54:15.067410 21301 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8978bd1c2254a8b80ba736378625821 [sys.catalog]: This master's current role is: LEADER
W20260806 22:54:15.067526 21313 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b8978bd1c2254a8b80ba736378625821: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 22:54:15.067586 21313 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 22:54:15.067795 21046 master_runner.cc:186] Error getting master registration for 127.28.218.253:45491: OK, instance_id { permanent_uuid: "720d2c909a504af894246cd0d7986856" instance_seqno: 1786056854845365 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 22:54:15.069984 21046 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 22:54:15.070303 21321 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:54:15.070966 21321 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:54:15.073339 21251 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 78ca88aa90374be9b1c08d7bd455fb07 [term 1 FOLLOWER]: Refusing update from remote peer b8978bd1c2254a8b80ba736378625821: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:54:15.073760 21314 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b8978bd1c2254a8b80ba736378625821 [LEADER]: Connected to new peer: Peer: permanent_uuid: "78ca88aa90374be9b1c08d7bd455fb07" member_type: VOTER last_known_addr { host: "127.28.218.252" port: 40877 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:54:15.075225 21287 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78ca88aa90374be9b1c08d7bd455fb07 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b8978bd1c2254a8b80ba736378625821. Latest consensus state: current_term: 1 leader_uuid: "b8978bd1c2254a8b80ba736378625821" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8978bd1c2254a8b80ba736378625821" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 34875 } } peers { permanent_uuid: "720d2c909a504af894246cd0d7986856" member_type: VOTER last_known_addr { host: "127.28.218.253" port: 45491 } } peers { permanent_uuid: "78ca88aa90374be9b1c08d7bd455fb07" member_type: VOTER last_known_addr { host: "127.28.218.252" port: 40877 } } }
I20260806 22:54:15.075336 21287 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78ca88aa90374be9b1c08d7bd455fb07 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:54:15.076198 21123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:54:15.067811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53086 (local address 127.28.218.254:34875)
0806 22:54:15.067847 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0806 22:54:15.067849 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:54:15.068599 (+ 750us) server_negotiation.cc:407] Connection header received
0806 22:54:15.068708 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:54:15.068708 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:54:15.068717 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:54:15.068749 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:54:15.069519 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:54:15.070206 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:54:15.071264 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:54:15.071561 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:54:15.071622 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:54:15.073155 (+ 1533us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:54:15.073156 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:54:15.073156 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:54:15.073163 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:54:15.074826 (+ 1663us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:54:15.075300 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:54:15.075300 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:54:15.075301 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:54:15.075365 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:54:15.075598 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:54:15.075599 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:54:15.075599 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:54:15.075734 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:54:15.075822 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 22:54:15.076019 (+ 197us) server_negotiation.cc:299] Negotiation successful
0806 22:54:15.076114 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 22:54:15.076920 21301 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8978bd1c2254a8b80ba736378625821 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b8978bd1c2254a8b80ba736378625821" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8978bd1c2254a8b80ba736378625821" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 34875 } } peers { permanent_uuid: "720d2c909a504af894246cd0d7986856" member_type: VOTER last_known_addr { host: "127.28.218.253" port: 45491 } } peers { permanent_uuid: "78ca88aa90374be9b1c08d7bd455fb07" member_type: VOTER last_known_addr { host: "127.28.218.252" port: 40877 } } }
I20260806 22:54:15.076994 21301 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8978bd1c2254a8b80ba736378625821 [sys.catalog]: This master's current role is: LEADER
I20260806 22:54:15.077557 21287 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78ca88aa90374be9b1c08d7bd455fb07 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b8978bd1c2254a8b80ba736378625821" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8978bd1c2254a8b80ba736378625821" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 34875 } } peers { permanent_uuid: "720d2c909a504af894246cd0d7986856" member_type: VOTER last_known_addr { host: "127.28.218.253" port: 45491 } } peers { permanent_uuid: "78ca88aa90374be9b1c08d7bd455fb07" member_type: VOTER last_known_addr { host: "127.28.218.252" port: 40877 } } }
I20260806 22:54:15.077631 21287 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78ca88aa90374be9b1c08d7bd455fb07 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:54:15.078017 21325 mvcc.cc:205] Tried to move back new op lower bound from 7315688878377644032 to 7315688878354821120. Current Snapshot: MvccSnapshot[applied={T|T < 7315688878377644032}]
I20260806 22:54:15.078140 21199 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.218.252" port: 40877 } attempt: 3
I20260806 22:54:15.078739 21326 mvcc.cc:205] Tried to move back new op lower bound from 7315688878377644032 to 7315688878354821120. Current Snapshot: MvccSnapshot[applied={T|T < 7315688878377644032}]
I20260806 22:54:15.079645 21321 catalog_manager.cc:1347] Generated new cluster ID: 6c04ba07af6d49cd911dd56247a70e12
I20260806 22:54:15.079691 21321 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:54:15.084215 21204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:54:15.068123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56636 (local address 127.28.218.253:45491)
0806 22:54:15.068163 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 22:54:15.068165 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:54:15.076928 (+ 8763us) server_negotiation.cc:407] Connection header received
0806 22:54:15.077032 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:54:15.077033 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:54:15.077045 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:54:15.077074 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:54:15.077730 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:54:15.078440 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:54:15.079436 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:54:15.079745 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:54:15.079788 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:54:15.081188 (+ 1400us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:54:15.081189 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:54:15.081190 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:54:15.081196 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:54:15.082868 (+ 1672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:54:15.083284 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:54:15.083285 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:54:15.083286 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:54:15.083351 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:54:15.083585 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:54:15.083587 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:54:15.083587 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:54:15.083727 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:54:15.083843 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0806 22:54:15.084052 (+ 209us) server_negotiation.cc:299] Negotiation successful
0806 22:54:15.084128 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
Aug 06 22:54:15 dist-test-slave-1xrh krb5kdc[21033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056854, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.218.253@KRBTEST.COM for kudu/127.28.218.252@KRBTEST.COM
I20260806 22:54:15.090487 21285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:54:15.068409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53082 (local address 127.28.218.252:40877)
0806 22:54:15.068445 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0806 22:54:15.068447 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:54:15.083251 (+ 14804us) server_negotiation.cc:407] Connection header received
0806 22:54:15.083348 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:54:15.083349 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:54:15.083360 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:54:15.083392 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:54:15.084065 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:54:15.084715 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:54:15.085682 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:54:15.085972 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:54:15.086010 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:54:15.087464 (+ 1454us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:54:15.087465 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:54:15.087465 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:54:15.087472 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:54:15.089262 (+ 1790us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:54:15.089641 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:54:15.089642 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:54:15.089643 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:54:15.089710 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:54:15.089920 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:54:15.089921 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:54:15.089921 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:54:15.090050 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:54:15.090119 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 22:54:15.090321 (+ 202us) server_negotiation.cc:299] Negotiation successful
0806 22:54:15.090391 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 22:54:15.094775 21203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:54:15.078375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.218.252:40877 (local address 127.0.0.1:53096)
0806 22:54:15.078454 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0806 22:54:15.078464 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:54:15.078504 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:54:15.085222 (+ 6718us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:54:15.085223 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:54:15.085370 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:54:15.085402 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:54:15.085403 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:54:15.086618 (+ 1215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:54:15.086619 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:54:15.087474 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:54:15.087477 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:54:15.088481 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:54:15.088482 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:54:15.088542 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:54:15.089228 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:54:15.089234 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:54:15.090710 (+ 1476us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:54:15.093033 (+ 2323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:54:15.093034 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:54:15.093035 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:54:15.093249 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:54:15.094174 (+ 925us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:54:15.094175 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:54:15.094175 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:54:15.094233 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:54:15.094505 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:54:15.094506 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:54:15.094556 (+ 50us) client_negotiation.cc:769] Sending connection context
0806 22:54:15.094599 (+ 43us) client_negotiation.cc:240] Negotiation successful
0806 22:54:15.094686 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63}
I20260806 22:54:15.095181 21331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:54:15.078569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53096 (local address 127.28.218.252:40877)
0806 22:54:15.084888 (+ 6319us) server_negotiation.cc:206] Beginning negotiation
0806 22:54:15.084890 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:54:15.084897 (+ 7us) server_negotiation.cc:407] Connection header received
0806 22:54:15.084915 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:54:15.084915 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:54:15.085124 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:54:15.085477 (+ 353us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:54:15.085506 (+ 29us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:54:15.086520 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:54:15.087561 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:54:15.088130 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:54:15.091202 (+ 3072us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:54:15.091248 (+ 46us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:54:15.091248 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:54:15.091249 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:54:15.091253 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:54:15.092936 (+ 1683us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:54:15.094028 (+ 1092us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:54:15.094028 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:54:15.094029 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:54:15.094092 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:54:15.094301 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:54:15.094301 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:54:15.094302 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:54:15.094417 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:54:15.094931 (+ 514us) server_negotiation.cc:1035] Waiting for connection context
0806 22:54:15.094971 (+ 40us) server_negotiation.cc:299] Negotiation successful
0806 22:54:15.095044 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6251,"thread_start_us":77,"threads_started":1}
I20260806 22:54:15.096390 21321 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:54:15.096501 21199 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 720d2c909a504af894246cd0d7986856: Bootstrap starting.
I20260806 22:54:15.097146 21199 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 720d2c909a504af894246cd0d7986856: Neither blocks nor log segments found. Creating new log.
I20260806 22:54:15.097414 21199 log.cc:824] T 00000000000000000000000000000000 P 720d2c909a504af894246cd0d7986856: Log is configured to *not* fsync() on all Append() calls
I20260806 22:54:15.097832 21321 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:54:15.104354 21199 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 720d2c909a504af894246cd0d7986856: No bootstrap required, opened a new log
I20260806 22:54:15.106281 21199 raft_consensus.cc:348] T 00000000000000000000000000000000 P 720d2c909a504af894246cd0d7986856 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8978bd1c2254a8b80ba736378625821" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 34875 } } peers { permanent_uuid: "720d2c909a504af894246cd0d7986856" member_type: VOTER last_known_addr { host: "127.28.218.253" port: 45491 } } peers { permanent_uuid: "78ca88aa90374be9b1c08d7bd455fb07" member_type: VOTER last_known_addr { host: "127.28.218.252" port: 40877 } }
I20260806 22:54:15.106417 21199 raft_consensus.cc:729] T 00000000000000000000000000000000 P 720d2c909a504af894246cd0d7986856 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 720d2c909a504af894246cd0d7986856, State: Initialized, Role: FOLLOWER
I20260806 22:54:15.106506 21199 consensus_queue.cc:260] T 00000000000000000000000000000000 P 720d2c909a504af894246cd0d7986856 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8978bd1c2254a8b80ba736378625821" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 34875 } } peers { permanent_uuid: "720d2c909a504af894246cd0d7986856" member_type: VOTER last_known_addr { host: "127.28.218.253" port: 45491 } } peers { permanent_uuid: "78ca88aa90374be9b1c08d7bd455fb07" member_type: VOTER last_known_addr { host: "127.28.218.252" port: 40877 } }
I20260806 22:54:15.106813 21199 sys_catalog.cc:564] T 00000000000000000000000000000000 P 720d2c909a504af894246cd0d7986856 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:54:15.107211 21169 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 720d2c909a504af894246cd0d7986856 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:54:15.108047 21169 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 720d2c909a504af894246cd0d7986856 [term 1 FOLLOWER]: Refusing update from remote peer b8978bd1c2254a8b80ba736378625821: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:54:15.108340 21329 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b8978bd1c2254a8b80ba736378625821 [LEADER]: Connected to new peer: Peer: permanent_uuid: "720d2c909a504af894246cd0d7986856" member_type: VOTER last_known_addr { host: "127.28.218.253" port: 45491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:54:15.108609 21170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b8978bd1c2254a8b80ba736378625821" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "720d2c909a504af894246cd0d7986856"
I20260806 22:54:15.109400 21170 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 720d2c909a504af894246cd0d7986856 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8978bd1c2254a8b80ba736378625821 in term 1.
W20260806 22:54:15.113370 21349 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 720d2c909a504af894246cd0d7986856: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 22:54:15.113428 21349 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 22:54:15.113528 21337 sys_catalog.cc:455] T 00000000000000000000000000000000 P 720d2c909a504af894246cd0d7986856 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b8978bd1c2254a8b80ba736378625821" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8978bd1c2254a8b80ba736378625821" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 34875 } } peers { permanent_uuid: "720d2c909a504af894246cd0d7986856" member_type: VOTER last_known_addr { host: "127.28.218.253" port: 45491 } } peers { permanent_uuid: "78ca88aa90374be9b1c08d7bd455fb07" member_type: VOTER last_known_addr { host: "127.28.218.252" port: 40877 } } }
I20260806 22:54:15.113603 21337 sys_catalog.cc:458] T 00000000000000000000000000000000 P 720d2c909a504af894246cd0d7986856 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:54:15.113649 21337 sys_catalog.cc:455] T 00000000000000000000000000000000 P 720d2c909a504af894246cd0d7986856 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b8978bd1c2254a8b80ba736378625821. Latest consensus state: current_term: 1 leader_uuid: "b8978bd1c2254a8b80ba736378625821" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8978bd1c2254a8b80ba736378625821" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 34875 } } peers { permanent_uuid: "720d2c909a504af894246cd0d7986856" member_type: VOTER last_known_addr { host: "127.28.218.253" port: 45491 } } peers { permanent_uuid: "78ca88aa90374be9b1c08d7bd455fb07" member_type: VOTER last_known_addr { host: "127.28.218.252" port: 40877 } } }
I20260806 22:54:15.113698 21337 sys_catalog.cc:458] T 00000000000000000000000000000000 P 720d2c909a504af894246cd0d7986856 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:54:15.113730 21337 sys_catalog.cc:455] T 00000000000000000000000000000000 P 720d2c909a504af894246cd0d7986856 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b8978bd1c2254a8b80ba736378625821" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8978bd1c2254a8b80ba736378625821" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 34875 } } peers { permanent_uuid: "720d2c909a504af894246cd0d7986856" member_type: VOTER last_known_addr { host: "127.28.218.253" port: 45491 } } peers { permanent_uuid: "78ca88aa90374be9b1c08d7bd455fb07" member_type: VOTER last_known_addr { host: "127.28.218.252" port: 40877 } } }
I20260806 22:54:15.113775 21337 sys_catalog.cc:458] T 00000000000000000000000000000000 P 720d2c909a504af894246cd0d7986856 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:54:15.114852 21335 mvcc.cc:205] Tried to move back new op lower bound from 7315688878466572288 to 7315688878354821120. Current Snapshot: MvccSnapshot[applied={T|T < 7315688878377644032}]
I20260806 22:54:15.115109 21329 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8978bd1c2254a8b80ba736378625821 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b8978bd1c2254a8b80ba736378625821" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8978bd1c2254a8b80ba736378625821" member_type: VOTER last_known_addr { host: "127.28.218.254" port: 34875 } } peers { permanent_uuid: "720d2c909a504af894246cd0d7986856" member_type: VOTER last_known_addr { host: "127.28.218.253" port: 45491 } } peers { permanent_uuid: "78ca88aa90374be9b1c08d7bd455fb07" member_type: VOTER last_known_addr { host: "127.28.218.252" port: 40877 } } }
I20260806 22:54:15.115217 21329 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8978bd1c2254a8b80ba736378625821 [sys.catalog]: This master's current role is: LEADER
I20260806 22:54:15.116883 21321 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b8978bd1c2254a8b80ba736378625821: Generated new TSK 0
I20260806 22:54:15.117084 21321 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 22:54:15.143205 21123 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:54:15.141659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53088 (local address 127.28.218.254:34875)
0806 22:54:15.141700 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0806 22:54:15.141703 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:54:15.142532 (+ 829us) server_negotiation.cc:407] Connection header received
0806 22:54:15.142643 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:54:15.142644 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:54:15.142656 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:54:15.142687 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:54:15.143114 (+ 427us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53088: BlockingRecv error: recv got EOF from 127.0.0.1:53088 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260806 22:54:15.144140 21204 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:54:15.142014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56650 (local address 127.28.218.253:45491)
0806 22:54:15.142052 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0806 22:54:15.142053 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:54:15.143619 (+ 1566us) server_negotiation.cc:407] Connection header received
0806 22:54:15.143715 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:54:15.143716 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:54:15.143727 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:54:15.143756 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:54:15.144083 (+ 327us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56650: BlockingRecv error: recv got EOF from 127.0.0.1:56650 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260806 22:54:15.145071 21331 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:54:15.142269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53112 (local address 127.28.218.252:40877)
0806 22:54:15.142305 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0806 22:54:15.142319 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 22:54:15.144572 (+ 2253us) server_negotiation.cc:407] Connection header received
0806 22:54:15.144668 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:54:15.144669 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:54:15.144680 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:54:15.144710 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:54:15.145015 (+ 305us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53112: BlockingRecv error: recv got EOF from 127.0.0.1:53112 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260806 22:54:15.147445 21204 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:54:15.146294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56664 (local address 127.28.218.253:45491)
0806 22:54:15.146342 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 22:54:15.146343 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:54:15.146880 (+ 537us) server_negotiation.cc:407] Connection header received
0806 22:54:15.147015 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:54:15.147016 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:54:15.147033 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:54:15.147060 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:54:15.147407 (+ 347us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56664: BlockingRecv error: recv got EOF from 127.0.0.1:56664 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
W20260806 22:54:15.150439 21204 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:54:15.149642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56666 (local address 127.28.218.253:45491)
0806 22:54:15.149673 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0806 22:54:15.149674 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:54:15.149934 (+ 260us) server_negotiation.cc:407] Connection header received
0806 22:54:15.150030 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:54:15.150031 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:54:15.150041 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:54:15.150070 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:54:15.150398 (+ 328us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56666: BlockingRecv error: recv got EOF from 127.0.0.1:56666 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260806 22:54:15.154186 21204 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:54:15.153472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56682 (local address 127.28.218.253:45491)
0806 22:54:15.153502 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0806 22:54:15.153503 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:54:15.153732 (+ 229us) server_negotiation.cc:407] Connection header received
0806 22:54:15.153822 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:54:15.153823 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:54:15.153833 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:54:15.153862 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:54:15.154150 (+ 288us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56682: BlockingRecv error: recv got EOF from 127.0.0.1:56682 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260806 22:54:15.159929 21204 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:54:15.159209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56692 (local address 127.28.218.253:45491)
0806 22:54:15.159251 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 22:54:15.159253 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:54:15.159472 (+ 219us) server_negotiation.cc:407] Connection header received
0806 22:54:15.159555 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:54:15.159556 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:54:15.159565 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:54:15.159590 (+ 25us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:54:15.159895 (+ 305us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56692: BlockingRecv error: recv got EOF from 127.0.0.1:56692 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
W20260806 22:54:15.169667 21204 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:54:15.168939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56706 (local address 127.28.218.253:45491)
0806 22:54:15.168971 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0806 22:54:15.168973 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:54:15.169196 (+ 223us) server_negotiation.cc:407] Connection header received
0806 22:54:15.169285 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:54:15.169286 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:54:15.169295 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:54:15.169339 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:54:15.169628 (+ 289us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56706: BlockingRecv error: recv got EOF from 127.0.0.1:56706 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260806 22:54:15.187680 21204 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:54:15.186809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56716 (local address 127.28.218.253:45491)
0806 22:54:15.186846 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0806 22:54:15.186848 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:54:15.187087 (+ 239us) server_negotiation.cc:407] Connection header received
0806 22:54:15.187254 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:54:15.187255 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:54:15.187265 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:54:15.187293 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:54:15.187632 (+ 339us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56716: BlockingRecv error: recv got EOF from 127.0.0.1:56716 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260806 22:54:15.221804 21204 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:54:15.221104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56720 (local address 127.28.218.253:45491)
0806 22:54:15.221146 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 22:54:15.221149 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:54:15.221276 (+ 127us) server_negotiation.cc:407] Connection header received
0806 22:54:15.221350 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:54:15.221351 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:54:15.221363 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:54:15.221434 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:54:15.221753 (+ 319us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56720: BlockingRecv error: recv got EOF from 127.0.0.1:56720 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260806 22:54:15.287667 21204 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:54:15.286908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56724 (local address 127.28.218.253:45491)
0806 22:54:15.286973 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0806 22:54:15.286976 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:54:15.287097 (+ 121us) server_negotiation.cc:407] Connection header received
0806 22:54:15.287202 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:54:15.287204 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:54:15.287215 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:54:15.287260 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:54:15.287588 (+ 328us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56724: BlockingRecv error: recv got EOF from 127.0.0.1:56724 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
W20260806 22:54:15.417569 21204 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:54:15.416921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56736 (local address 127.28.218.253:45491)
0806 22:54:15.416990 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0806 22:54:15.416992 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:54:15.417005 (+ 13us) server_negotiation.cc:407] Connection header received
0806 22:54:15.417066 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:54:15.417067 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:54:15.417078 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:54:15.417147 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:54:15.417475 (+ 328us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56736: BlockingRecv error: recv got EOF from 127.0.0.1:56736 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
W20260806 22:54:15.675594 21204 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:54:15.674845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56746 (local address 127.28.218.253:45491)
0806 22:54:15.674916 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0806 22:54:15.674918 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:54:15.674958 (+ 40us) server_negotiation.cc:407] Connection header received
0806 22:54:15.675018 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:54:15.675019 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:54:15.675029 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:54:15.675081 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:54:15.675514 (+ 433us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56746: BlockingRecv error: recv got EOF from 127.0.0.1:56746 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 22:54:16.029999 21298 catalog_manager.cc:1259] Loaded cluster ID: 6c04ba07af6d49cd911dd56247a70e12
I20260806 22:54:16.030082 21298 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 78ca88aa90374be9b1c08d7bd455fb07: loading cluster ID for follower catalog manager: success
I20260806 22:54:16.031265 21298 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 78ca88aa90374be9b1c08d7bd455fb07: acquiring CA information for follower catalog manager: success
I20260806 22:54:16.031629 21298 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 78ca88aa90374be9b1c08d7bd455fb07: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 22:54:16.113870 21349 catalog_manager.cc:1259] Loaded cluster ID: 6c04ba07af6d49cd911dd56247a70e12
I20260806 22:54:16.114068 21349 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 720d2c909a504af894246cd0d7986856: loading cluster ID for follower catalog manager: success
I20260806 22:54:16.114864 21349 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 720d2c909a504af894246cd0d7986856: acquiring CA information for follower catalog manager: success
I20260806 22:54:16.115231 21349 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 720d2c909a504af894246cd0d7986856: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 22:54:16.191355 21425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:54:16.188756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56752 (local address 127.28.218.253:45491)
0806 22:54:16.188884 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0806 22:54:16.188886 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:54:16.188895 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:54:16.188924 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:54:16.188925 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:54:16.188976 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:54:16.189066 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:54:16.189227 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:54:16.189787 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:54:16.190514 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:54:16.190737 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:54:16.190796 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:54:16.190932 (+ 136us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:54:16.191062 (+ 130us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:54:16.191168 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0806 22:54:16.191191 (+ 23us) server_negotiation.cc:299] Negotiation successful
0806 22:54:16.191225 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":76,"threads_started":1}
I20260806 22:54:16.195492 21431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:54:16.193148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53124 (local address 127.28.218.252:40877)
0806 22:54:16.193272 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0806 22:54:16.193274 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:54:16.193283 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:54:16.193332 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:54:16.193332 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:54:16.193367 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:54:16.193457 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:54:16.193553 (+ 96us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:54:16.194020 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:54:16.194724 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:54:16.194918 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:54:16.195003 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:54:16.195163 (+ 160us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:54:16.195258 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:54:16.195330 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 22:54:16.195381 (+ 51us) server_negotiation.cc:299] Negotiation successful
0806 22:54:16.195401 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":55,"threads_started":1}
I20260806 22:54:16.199887 21425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:54:16.197246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56754 (local address 127.28.218.253:45491)
0806 22:54:16.197296 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 22:54:16.197298 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:54:16.197487 (+ 189us) server_negotiation.cc:407] Connection header received
0806 22:54:16.197514 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:54:16.197515 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:54:16.197526 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:54:16.197572 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:54:16.197710 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:54:16.198419 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:54:16.199197 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:54:16.199422 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:54:16.199502 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:54:16.199635 (+ 133us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:54:16.199664 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:54:16.199714 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 22:54:16.199783 (+ 69us) server_negotiation.cc:299] Negotiation successful
0806 22:54:16.199813 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260806 22:54:16.200357 21431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:54:16.197512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53126 (local address 127.28.218.252:40877)
0806 22:54:16.197544 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0806 22:54:16.197546 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:54:16.197606 (+ 60us) server_negotiation.cc:407] Connection header received
0806 22:54:16.197676 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:54:16.197677 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:54:16.197687 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:54:16.197739 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:54:16.197878 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:54:16.198318 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:54:16.199461 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:54:16.199768 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:54:16.199806 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:54:16.200022 (+ 216us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:54:16.200048 (+ 26us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:54:16.200073 (+ 25us) server_negotiation.cc:1035] Waiting for connection context
0806 22:54:16.200298 (+ 225us) server_negotiation.cc:299] Negotiation successful
0806 22:54:16.200315 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260806 22:54:16.200419 21438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:54:16.197124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53090 (local address 127.28.218.254:34875)
0806 22:54:16.197235 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0806 22:54:16.197237 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:54:16.197246 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:54:16.197295 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:54:16.197296 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:54:16.197348 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:54:16.197438 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:54:16.197609 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:54:16.198395 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:54:16.199448 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:54:16.199788 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:54:16.199847 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:54:16.200074 (+ 227us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:54:16.200172 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:54:16.200249 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 22:54:16.200313 (+ 64us) server_negotiation.cc:299] Negotiation successful
0806 22:54:16.200336 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89,"thread_start_us":66,"threads_started":1}
I20260806 22:54:16.201902 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 21046
I20260806 22:54:16.208895 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 21127
I20260806 22:54:16.214537 29547 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQja9Us/build/release/bin/kudu with pid 21207
2026-08-06T22:54:16Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (2792 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (5626 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (269434 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST